4 ms·

Seems easy to defeat with frequency bounds checks. But presumably there exist adversarial sounds that to a human seem like music, or gibberish, or a different

by sjbase 9y ago

Seems easy to defeat with frequency bounds checks. But presumably there exist adversarial sounds that to a human seem like music, or gibberish, or a different phrase, but sound like "pay joe 1000 dollars" to the machine. The sonic equivalent to these: https://arxiv.org/abs/1703.08603 https://arxiv.org/abs/1703.08603

TeMPOraL 9y ago
Not sure what you mean by "frequency bounds check", but if you mean a bandpass filter, then it is not going to save you. The attack, as I understand the paper, works by having the AM-encoded ultrasound demodulate to the audible frequencies at the microphone, before frequency filtering and ADC. This would imply you could use this method to inject voice even to simple recording software (or even analog hardware!) - it's not limited to machine-learning voice recognition systems! The paper describes possible ways to mitigate the attack, though. One of them is to let the whole signal pass, and then try to detect ultrasonic AM in software, demodulate it and subtract from the audible frequencies. This is pretty much correcting for the physical structure of the microphone in software.
- sjbase 9y ago
  Wow, yeah you're right. I gave the feasibility analysis a more thorough read and it seems like they're exploiting the amplifier inside the mic itself to down-convert the frequency on portions of the signal. I don't fully understand what nonlinearity is being taken advantage of, but to as an acoustics layman this seems pretty impressive!
  - AstralStorm 9y ago
    Which means they're not using the membrane but the crummy amplifier electronics. Fixed with one or two capacitors. Alternatively they're using almost deadly signal levels to overdrive a microphone at a distance.
- AstralStorm 9y ago
  I'd say it would require extremely good placement of the emitter and knowledge of exact microphone mechanics to pull it off anywhere else than in the lab. I'd be impressed if it even worked in a lab.

by sjbase 9y ago

TeMPOraL 9y ago

Not sure what you mean by "frequency bounds check", but if you mean a bandpass filter, then it is not going to save you. The attack, as I understand the paper, works by having the AM-encoded ultrasound demodulate to the audible frequencies at the microphone, before frequency filtering and ADC. This would imply you could use this method to inject voice even to simple recording software (or even analog hardware!) - it's not limited to machine-learning voice recognition systems! The paper describes possible ways to mitigate the attack, though. One of them is to let the whole signal pass, and then try to detect ultrasonic AM in software, demodulate it and subtract from the audible frequencies. This is pretty much correcting for the physical structure of the microphone in software.

sjbase 9y ago
Wow, yeah you're right. I gave the feasibility analysis a more thorough read and it seems like they're exploiting the amplifier inside the mic itself to down-convert the frequency on portions of the signal. I don't fully understand what nonlinearity is being taken advantage of, but to as an acoustics layman this seems pretty impressive!
- AstralStorm 9y ago
  Which means they're not using the membrane but the crummy amplifier electronics. Fixed with one or two capacitors. Alternatively they're using almost deadly signal levels to overdrive a microphone at a distance.
AstralStorm 9y ago
I'd say it would require extremely good placement of the emitter and knowledge of exact microphone mechanics to pull it off anywhere else than in the lab. I'd be impressed if it even worked in a lab.