11 ms·

Most phone now have some kind of lock screen, which makes it pretty difficult to get to the butt dialing stage. Will speech command recognition get to that stag

Most phone now have some kind of lock screen, which makes it pretty difficult to get to the butt dialing stage. Will speech command recognition get to that stage? The main reason I won't have any of those products in my house is because of that. I'd much rather have a confirmation of some kind before the system takes action. "...random talking..." Assistant "I am recording" Me: "Stop recording"

modzu 8y ago
some kind of acoustic biometrics would be helpful here (ie respond only to account holder, or disable some actions for others) along with better heuristic recognition of directives thats not foolproof but much better than what we have now, and i think we are pretty close
- edwhitesell 8y ago
  Agreed, but I think vocal biometrics is a significantly more difficult problem to solve when even the best speech recognition still has issues like this. Phonetics is hard. Especially with ambient noise, echo and such. I had a conversation with one of the speech engineers when I worked at a speech recognition company and the level of detailed problems to solve was impressive. Totally made sense after talking about it, but things I would not have thought about before. I'd imagine the next thing to come in this area that would really make an improvement is an "on person" microphone. Maybe it's a pen in your pocket, or some kind of vibration detection (that could pick up the wearers voice), that would then allow some improvemnts in the domain of "who is talking" and how well the voice is processed.
  - jwandborg 8y ago
    And once you've managed to achieve perfect biometrics your next task is to prevent replay attacks. An alternative, "wireless" approach to near-perfect security would be to invent-plement some kind of vocal, human-executable GPG/TLS.
    - vidarh 8y ago
      You don't really need perfect biometrics for it to be useful, or even biometrics at all. A "my human just spoke in this room, and here's a signature of what the fleshbag said" broadcast from a microphone to let devices within a very short range to get confirmation of which person spoke if they picked up a command, coupled with pairing your speaker to your devices to give the notification from your mic more authority would already help substantially with most of the issues people have. It wouldn't stop a determined adversary, but for most people it's not a determined adversary that is the problem. It doesn't even need to get good audio - just enough to give a bit of an indication that what the device picked up was me talking and not random noise and ideally some way to somewhat correlate it to the audio the device picked up to give it an indication it was me it heard. It'd also give you the option of setting the devices to require confirmation for certain types of orders if they were not confirmed by an authorised device, or if they were not confirmed by a device (so you could let people present give instructions but not some random joker on voice chat in your online game for example). If we could get support for that into e.g. a watch, it'd be very much useful.
  - namibj 8y ago
    The vibration detection is commercially available, in the 3-digit $ range, but it's unfortunately hideous as it needs some mild force against the throat and thus requires a very high neckline or a scarf. The potential alternative, Subvocal recognition, suffers from the need to have nerve/muscle-sensing electrodes on/around the throat, which have poor long-term bio compatibility (think >10h/day, on average) and the alternative, implants, suffer from growing out if a wire sticks out, at least as far as I know. One might be able to implant special electrodes the skin would not reject, but I don't know of any suitable material. The obvious benefit is that you use your normal speaking logic, just stop at the point where you actually move your throat (afaik), and don't modulate with your mouth or lungs. It's silent for anyone around, so one could, theoretically, call someone while sitting inside a meeting, and hear both speakers, while being able to selectively talk to the other end of the phone line. Both of these technologies work in nightclubs and fighter jets. I assume they are very high SNR, as far as ambient noise is concerned. The subvocal one might just be a phonetic/intonation input, which then requires voice synthesis if actual voice is the goal.
- jwandborg 8y ago
  I agree, it would probably not even be close to foolproof. I believe any implementation of security through acoustic biometrics would be vulnerable to replay attacks. Systems to reproduce acoustics with high fidelity are commonplace - You might be using the output component of such a system right now if you're listening to music. You could make the Assistant remember the exact fingerprints of all previous activation phrases and only trust you if it was original. This could be circumvented if you spoke the activation phrase at any point where your assistant could not hear you, for example to another Assistant of the same brand.
  - murjinsee 8y ago
    Wouldn't modifying the sample slightly, like lowering pitch by a few cents or stretching parts, also make it seem original? Audio is definitely too easy to spoof for it to be a security method IMO.
  - modzu 8y ago
    i meant biometrics merely as a UX improvement, ie, to help prevent the device from responding to the wrong thing "accidentally" it may have a place in security as well but i can only see it as part of a much more holistic model
- ClassyJacket 8y ago
  "respond only to account holder, or disable some actions for others" This is exactly how the Google Home works right now.
cookiecaper 8y ago
On the Echo at least, there's an option to have the device make a thump-type sound when it's activated. There's also a light ring that activates out of the box and shows where it believes the audio is coming from.
- vidarh 8y ago
  I tested mine right now, and I can't see any correlation between the light ring and my location relative to it.
  - sqlacid 8y ago
    could be picking up a bounced audio if it close to a wall or other surface.
    - vidarh 8y ago
      Maybe, but if it can't handle that, it's kinda pointless to try to detect direction - all the places in my house where I'd be likely to want to place it are <0.5m from a wall or less; I'm certainly not going to place them in the middle of a room.
acranox 8y ago
You'd be amazed at my dad's ability to butt dial people with an iPhone. It's possible. Like multiple times per week possible.
- cicero 8y ago
  I always press the lock button before I put the phone back in my pocket, but I've seen people put the phone back in their pocket or purse unlocked. If you have the phone app already running, butt dialing is very likely to occur.
- Piskvorrr 8y ago
  Lock doesn't have to be swipe-to-open. I'd be amazed if he could buttprint-unlock ;)
  - krylon 8y ago
    http://anuscan.com/ http://anuscan.com/ It might be a bit awkward at first to do that in front of strangers, but once it catches on, that will wear off. ;-)
- fmj 8y ago
  My friend does this fairly often. He'll finish a call, and turn off his screen, but he has a tendency to to hit the fingerprint scanner as he's putting it into his pocket due to the design of the case and the way he holds it.
  - edwhitesell 8y ago
    Some Androids (like mine) won't lock for a few seconds after turning off the screen. It's a nice feature when you press to turn off the screen, and quickly realize you need to do something else. Just turn on the screen and it's still unlocked. Entering the pattern after it's locked is pretty much impossible though.
    - mehrdadn 8y ago
pishpash 8y ago
Yeah, a digital assistant could (should) verbalize "uh huh, yes, i'm listening, interesting, hmm" like a real, nosy human. Then you get a chance to say, "go away, alexa".
- vidarh 8y ago
  The Echo devices will light up when listening, and can also given audible warning. But it'd certainly be good with more feedback.
powercow 8y ago
as long as you remember to lock the phone before butt pocketing it. I still butt dial today but its mainly right after i talked to someone.
perl4ever 8y ago
I dropped my phone in the gym and it somehow typed garbage all over the Notes app where I keep my grocery list and record of what I did in the gym last... This sort of thing makes me nostalgic for the old days of explicit "Save" commands in every application.
- InitialLastName 8y ago
  I'll settle for a reasonable "undo" feature.
  - perl4ever 8y ago
    Undo seems to be a concept in Android apps sometimes, but I'm not aware of a universal way to access it, like Ctrl-Z. And once you have Undo, it's nice if you always have Redo, in case you Undo once too many.
jwalton 8y ago
I will have one of these in my house when it doesn't send recordings of me to the Internet in the first place. It can do the voice recognition locally. I'm actually really interested in having one of these in my home, I'm also actually really not interested in it being cloud based at all.

modzu 8y ago

some kind of acoustic biometrics would be helpful here (ie respond only to account holder, or disable some actions for others) along with better heuristic recognition of directives thats not foolproof but much better than what we have now, and i think we are pretty close

edwhitesell 8y ago
Agreed, but I think vocal biometrics is a significantly more difficult problem to solve when even the best speech recognition still has issues like this. Phonetics is hard. Especially with ambient noise, echo and such. I had a conversation with one of the speech engineers when I worked at a speech recognition company and the level of detailed problems to solve was impressive. Totally made sense after talking about it, but things I would not have thought about before. I'd imagine the next thing to come in this area that would really make an improvement is an "on person" microphone. Maybe it's a pen in your pocket, or some kind of vibration detection (that could pick up the wearers voice), that would then allow some improvemnts in the domain of "who is talking" and how well the voice is processed.
- jwandborg 8y ago
  And once you've managed to achieve perfect biometrics your next task is to prevent replay attacks. An alternative, "wireless" approach to near-perfect security would be to invent-plement some kind of vocal, human-executable GPG/TLS.
  - vidarh 8y ago
    You don't really need perfect biometrics for it to be useful, or even biometrics at all. A "my human just spoke in this room, and here's a signature of what the fleshbag said" broadcast from a microphone to let devices within a very short range to get confirmation of which person spoke if they picked up a command, coupled with pairing your speaker to your devices to give the notification from your mic more authority would already help substantially with most of the issues people have. It wouldn't stop a determined adversary, but for most people it's not a determined adversary that is the problem. It doesn't even need to get good audio - just enough to give a bit of an indication that what the device picked up was me talking and not random noise and ideally some way to somewhat correlate it to the audio the device picked up to give it an indication it was me it heard. It'd also give you the option of setting the devices to require confirmation for certain types of orders if they were not confirmed by an authorised device, or if they were not confirmed by a device (so you could let people present give instructions but not some random joker on voice chat in your online game for example). If we could get support for that into e.g. a watch, it'd be very much useful.
- namibj 8y ago
  The vibration detection is commercially available, in the 3-digit $ range, but it's unfortunately hideous as it needs some mild force against the throat and thus requires a very high neckline or a scarf. The potential alternative, Subvocal recognition, suffers from the need to have nerve/muscle-sensing electrodes on/around the throat, which have poor long-term bio compatibility (think >10h/day, on average) and the alternative, implants, suffer from growing out if a wire sticks out, at least as far as I know. One might be able to implant special electrodes the skin would not reject, but I don't know of any suitable material. The obvious benefit is that you use your normal speaking logic, just stop at the point where you actually move your throat (afaik), and don't modulate with your mouth or lungs. It's silent for anyone around, so one could, theoretically, call someone while sitting inside a meeting, and hear both speakers, while being able to selectively talk to the other end of the phone line. Both of these technologies work in nightclubs and fighter jets. I assume they are very high SNR, as far as ambient noise is concerned. The subvocal one might just be a phonetic/intonation input, which then requires voice synthesis if actual voice is the goal.
jwandborg 8y ago
I agree, it would probably not even be close to foolproof. I believe any implementation of security through acoustic biometrics would be vulnerable to replay attacks. Systems to reproduce acoustics with high fidelity are commonplace - You might be using the output component of such a system right now if you're listening to music. You could make the Assistant remember the exact fingerprints of all previous activation phrases and only trust you if it was original. This could be circumvented if you spoke the activation phrase at any point where your assistant could not hear you, for example to another Assistant of the same brand.
- murjinsee 8y ago
  Wouldn't modifying the sample slightly, like lowering pitch by a few cents or stretching parts, also make it seem original? Audio is definitely too easy to spoof for it to be a security method IMO.
- modzu 8y ago
  i meant biometrics merely as a UX improvement, ie, to help prevent the device from responding to the wrong thing "accidentally" it may have a place in security as well but i can only see it as part of a much more holistic model
ClassyJacket 8y ago
"respond only to account holder, or disable some actions for others" This is exactly how the Google Home works right now.

cookiecaper 8y ago

On the Echo at least, there's an option to have the device make a thump-type sound when it's activated. There's also a light ring that activates out of the box and shows where it believes the audio is coming from.

vidarh 8y ago
I tested mine right now, and I can't see any correlation between the light ring and my location relative to it.
- sqlacid 8y ago
  could be picking up a bounced audio if it close to a wall or other surface.
  - vidarh 8y ago
    Maybe, but if it can't handle that, it's kinda pointless to try to detect direction - all the places in my house where I'd be likely to want to place it are <0.5m from a wall or less; I'm certainly not going to place them in the middle of a room.

acranox 8y ago

You'd be amazed at my dad's ability to butt dial people with an iPhone. It's possible. Like multiple times per week possible.

cicero 8y ago
I always press the lock button before I put the phone back in my pocket, but I've seen people put the phone back in their pocket or purse unlocked. If you have the phone app already running, butt dialing is very likely to occur.
Piskvorrr 8y ago
Lock doesn't have to be swipe-to-open. I'd be amazed if he could buttprint-unlock ;)
- krylon 8y ago
  http://anuscan.com/ http://anuscan.com/ It might be a bit awkward at first to do that in front of strangers, but once it catches on, that will wear off. ;-)
fmj 8y ago
My friend does this fairly often. He'll finish a call, and turn off his screen, but he has a tendency to to hit the fingerprint scanner as he's putting it into his pocket due to the design of the case and the way he holds it.
- edwhitesell 8y ago
  Some Androids (like mine) won't lock for a few seconds after turning off the screen. It's a nice feature when you press to turn off the screen, and quickly realize you need to do something else. Just turn on the screen and it's still unlocked. Entering the pattern after it's locked is pretty much impossible though.
  - mehrdadn 8y ago

pishpash 8y ago

Yeah, a digital assistant could (should) verbalize "uh huh, yes, i'm listening, interesting, hmm" like a real, nosy human. Then you get a chance to say, "go away, alexa".

vidarh 8y ago
The Echo devices will light up when listening, and can also given audible warning. But it'd certainly be good with more feedback.

powercow 8y ago

as long as you remember to lock the phone before butt pocketing it. I still butt dial today but its mainly right after i talked to someone.

perl4ever 8y ago

I dropped my phone in the gym and it somehow typed garbage all over the Notes app where I keep my grocery list and record of what I did in the gym last... This sort of thing makes me nostalgic for the old days of explicit "Save" commands in every application.

InitialLastName 8y ago
I'll settle for a reasonable "undo" feature.
- perl4ever 8y ago
  Undo seems to be a concept in Android apps sometimes, but I'm not aware of a universal way to access it, like Ctrl-Z. And once you have Undo, it's nice if you always have Redo, in case you Undo once too many.

jwalton 8y ago

I will have one of these in my house when it doesn't send recordings of me to the Internet in the first place. It can do the voice recognition locally. I'm actually really interested in having one of these in my home, I'm also actually really not interested in it being cloud based at all.