6 ms·
Voice Interfaces
- antidaily 12y agoThe reason current voice interfaces suck is because they force the speaker to consciously enter a “voice” mode and then create context around the action they want the computer to perform Isnt Google testing this right now? ex http://i.imgur.com/fJrQZ0H.jpg http://i.imgur.com/fJrQZ0H.jpg
- jon-wood 12y agoThey still require the utterly ridiculous "Ok, Google" activation though. I know its irrational, but I really don't want to have to use phrases I would never use in day to day life to communicate with my computer.
- jeffgreco 12y agoIsn't that the whole point of an obscure catchword? To avoid the computer intercepting normal conversation?
- smacktoward 12y agoSure, but that's just an artifact of the crude capabilities of current speech recognition. A sufficiently advanced speech recognition engine wouldn't need you to insert a stopword to know that you're talking to it. It would just... know.
- sejje 12y agoThat's a bit strange. I don't always know whether people are talking to me, and I wouldn't say my speech recognition is crude. Why don't they just allow for naming a device? You'd use names in your everyday conversation.
- untog 12y agoSometimes I, as a human being, don't know if someone is talking to me unless they preface their statement with my name. How are computers supposed to be any better?
- jodrellblank 12y agoLong term, because it can monitor way more world state and be designed with as much processing power as necessary; you have one brain and don't want to spend 100% of your time working out if someone is talking to you.
- drdaeman 12y agoJust wonder whenever someone had experimented with use of constructed or just uncommon languages for such purposes. Such approach is certainly not consumer-friendly, but fun nonetheless. Say, a geeky enough person could use Esperanto or Lojban to control always-listening home automation without worrying too much it'll accidentally get unintended commands. Or, maybe, not a general purpose conlang, but a specially constructed one tailored just to quickly express the necessary concepts to machine.
- canthonytucci 12y agoNot exactly this, but iOS and google both have voice mode shortcuts you can use "Hey Siri" (which i think only works when the device is plugged in) and "Ok, Google". The problem I see is users getting frustrated with false positives - having the device do things they are not expecting. Pretty much the experience someone has when they first try vi or ed.
- jmonegro 12y agoYou can open apps with Siri on iOS.
- trishume 12y agoOne source of this problem is that only Google and Nuance have voice recognition engines that are any good, and they are very closed up (unless you pay tons of money to Nuance like Apple does for Siri). Most developer's only option for voice technology is to use Nuance's API which requires uploading the voice sample and waiting for a response, this is nowhere near fast enough for pleasant interaction. Things will only get better when Apple, Google and Microsoft open up really high quality on-device speech APIs for their operating systems.
- cbr 12y agoIn addition to using voice actions to launch activities, you can also call the system's built-in Speech Recognizer activity to obtain speech input from users. This is useful to obtain input from users and then process it, such as doing a search or sending it as a message. https://developer.android.com/training/wearables/apps/voice.html#FreeFormSpeech https://developer.android.com/training/wearables/apps/voice....
- AVTizzle 12y agoOf the examples Dustin offers here, the ones that have voice replacing typing make the most sense: - When I’m inputting my home address in a web browser (on mobile or desktop), I should be able to tap the “State” dropdown and just say “California” and it have it select that option for me. - When I highlight the browser address bar, I should be able to just say “The Economist” and have it automatically find the address in my favorites and go there. ... - When I click the “To” field in a mail app or in Gmail, I should be able to just say a person’s name and have it fill in automatically (and maybe show me a dropdown to select which email address to send to). Dustin's examples where the voice recognition replace a series of swipes and or taps seem, to me, a bit frivolous: - When I open to the home screen on my phone, I should be able to just say “Instagram” and have that app open. - On iOS, when I get a notification that covers the top of the screen, I should be able to just say “ignore” and have the notification instantly disappear. I know exactly where Instagram is on my phone. It's a swipe and a tap. And to ignore a notification on iOS, all one has to do is swipe up on it. The incremental time spent vs voice control seems negligible, and not something I'd fret about as a user nor a UX designer.
- smacktoward 12y ago> Because they are still so frustratingly limited, Siri and Google Now are simply not yet ready to exist. In fairness, Google Now's pitch is a lot more about location-awareness than it is about speech recognition, innit?
- dragonwriter 12y ago> In fairness, Google Now's pitch is a lot more about location-awareness than it is about speech recognition, innit Google Now's pitch is a lot more about all-around awareness -- not just "location" awareness. Voice isn't really even part of it (the voice actions are all part of the Google app, which Now integrates, but they aren't really part of Now -- and actually predate it.)
- untog 12y agoWhen I open to the home screen on my phone, I should be able to just say “Instagram” and have that app open. Is it me, or is the benefit of that extremely small? I can move my finger and tap the 'Instagram' icon quicker than I can say "Instagram". Never mind that the app itself is highly visual, so there's no point launching it without already looking at the screen and holding the phone to your hand. When I’m inputting my home address in a web browser (on mobile or desktop), I should be able to tap the “State” dropdown and just say “California” and it have it select that option for me. Or it should just already know your home address. That's why I don't think Google Now should be included in this criticism - the idea of it is to preempt any need for you to issue voice commands, because it's already presented you with the information you need. On iOS, when I get a notification that covers the top of the screen, I should be able to just say “ignore” and have the notification instantly disappear. I really don't think voice is a great interface at all, because it involves invading other people's personal space. I don't want to be sat on the train with people saying "Instagram","Facebook", "ignore", "ignore" all around me. I don't see what's so bad about the current interfaces, nor do I see anything in this post that is particularly new or insightful.
- sergiotapia 12y ago>Is it me, or is the benefit of that extremely small? I can move my finger and tap the 'Instagram' icon quicker than I can say "Instagram". That's because you're picturing it as the slow clunky voice detection we have today. Imagine just picking up your phone and saying naturally: "untog's instagram" and it loads up, no touching no prompting.
- untog 12y agoEr, I still don't understand how that is quicker. You mean I get to skip unlocking my keypad? Apple's fingerprint scanner thing is a much better solution to that problem. And how often does that get triggered by accident? I saw to a friend, "did you see my photo on Instagram", and my phone loads the app. I'm sorry, but voice is an awful interface. And it has been for decades, ever since people first proposed the idea of office workers sitting around dictating their work. It never happened, and that's a good thing.
- serve_yay 12y agoOkay? Noted? I'm not sure how much there is to respond to here.
- kajecounterhack 12y agoI think his post demonstrates a poor understanding of the state of voice technology. > That being said, current voice recognition technology is incredibly good at certain things. It’s great at detecting and transcribing words, listening for specific commands, and making matches against expected inputs. Current state of the art is not actually great at transcription or detection, in fact only Google and Apple's algos are any good and they both involve sending all your voice recordings to their servers where huge models are used to transcribe, meaning lag times and it's not efficient to always be sending every minute of audio to them. If you want to do continuous listening or use local voice transcription it's possible but you are limited by your hardware. Listening for activation words is also a hard problem. In fact, Moto X has a chip designed just to listen for "Ok Google" so clearly you see some of the problems: 1. it uses lots of power, 2. it has no flexibility. Other examples he gives assumes the phone won't intercept outside noise or will differentiate between voices. This is unfortunately not the case -- you could maybe apply a training and have a custom voice model but it's still just an unrealistic idea at this point. > The reason current voice interfaces suck is because they force the speaker to consciously enter a “voice” mode and then create context around the action they want the computer to perform. This makes no sense; the computer should just always be listening for potential commands within the context of whatever the user is doing. Yes, this is true. But it's all side-effect because of power, bandwidth, and processing-power constraints given today's algos / models. Today's voice processing algos rely on lots of data. And also the fact that voices aren't differentiated right now and it'd be havoc if everyone were setting off each others' phones.
- eghad 12y agoYour post is a bit ironic seeing as the Moto X does differentiate between voice (if you're good at imitating someone's voice you can set it off, but sending a command is unlikely) and the power usage is not as bad as you assume (it's got average battery life but can easily make it through the day), but I will agree that the "Ok Google Now" being inflexible is a niggling point. The new Moto X is always listening as well, but allows you to change the trigger phrase via some magic they've worked out.
- 12y ago
- themodelplumber 12y ago> Siri and Google Now are simply not yet ready to exist After reading the article: Dustin's ideas are simply not ready to exist. And somehow he hand-waves toward the ever-enlarging group that uses Siri and Google Now all the time. Sports, weather, directions, etc.
- VLM 12y ago"ever-enlarging group that uses Siri and Google Now" I was curious where we are on the stereotypical fad graph, the upswing, the plateau of disillusionment, the decline, whatever. Turns out its VERY hard to find usage stats. "In the wild" I've never seen any human being use siri or now, other than fooling around a couple years back when it was new. My wife and I both have Now equipped phones and its mostly just an annoyance when it detects an upward swipe. Anecdotally I find voice UI much like having printer support on my phone. Something I imagine would be insanely useful, so of course I have an app that can print to my printer. What an amazing network of possibilities. But in practice I never use it. Ever. Once to verify it works. Yup, it works. Never printed anything again. To some extent this is just "owning a laser printer" in general, which I really don't need and will not replace when it eventually breaks.
- justinsb 12y agoI think this just shows that Siri has fallen behind Android. On Android: "Who invented the light bulb?": lists the 3 inventors "Open web-page The Economist": goes to theeconomist.com "Launch Instagram": launches instagram app "Send email to <person>": starts an email to <person> There are lots more, though you do have to guess/remember the magic words if you don't want to do a web-search. And yes, it is annoying that you can't enter California in a drop-down by voice (but really, your browser should be auto-completing that for you anyway). But the hard problems have been solved, and I see a bright future here.
- refulgentis 12y agoWith the exception of the light bulb query, these work just fine with iOS too – the thrust of his post is that these actions should be available without having to specifically request a voice interaction mode to be turned on, which Android requires as well.
- justinsb 12y agoI certainly take your point; that would be awesome. And thanks for the iOS information! It was more confused by the original article though; he seemed to be suggesting that you would click on a particular UI component first: Want to open an webpage? Go to the home screen, then say "Web Browser", then click on the browser bar, then say 'The Economist'. But why touch on small areas of the screen at all, if Siri is just a button-hold away and takes you right there? I think the take-home message is that many of those UI components could go away - we don't need a browser bar, we don't need a home screen, if we choose to use voice instead.
- jodrellblank 12y agoThe whole point of the article is that the button hold is bad because Siri has no context to understand what you are about to say, isn't very intelligent, and that means you end up making up weird phrasings to try and indicate that you want the website "the Economist" to load in a browser, rather than directions to The Economist office building, or facts about The Economist website traffic from wolfram Alpha, or to open The Economist latest issue in iBooks ... By being in a browser and selecting the URL bar, the context is narrowed from "anything you can say in English" to "a website, if it's not a website, search for it". By dropping a dropdown, the context is "one item from this list". N.b. he also talks about desktops, not just mobile.
- jjcm 12y agoI worked with TellMe (a voice reco company) right after they were acquired by Microsoft. My job was mainly to work on VUIs (voice user interfaces) for the Windows Phone, Xbox, and Cortana during its early days. There are two things blocking voice interfaces from being more prevalent. The first is an action word. As you said yourself, all voice systems today require the user to enter in to a voice mode before actions can be recognized. It's sadly a necessity as devices today can't differentiate between when they're being talked to and when a user is talking to someone else. Even if you only allow voice commands within a specific context (i.e. an alert pops up, the user says "ignore", the pop up goes away), you're playing a very dangerous game. Take this scenario for instance: Bob on his computer: "Have you seen this cat video? Look at how the cute kitty cat will completely..." Phone: POP UP WITH ALERT ALL YOUR ICLOUD NUDES ARE BEING STOLEN Bob on his computer: "...ignore..." Phone: pop up goes away, never seen by Bob (unlike his nudes, which are now everywhere) Bob on his computer "...everything around him! I would never ignore things that were important." See the problem there? While it is a slim chance of happening, the effects of misinterpreting a command can be dire. The potential negatives outweigh the convenience. Maybe if everyone actively used voice commands, the net sum would be positive despite it destroying a select few peoples' lives, but that brings us to our other problem: Voice commands are socially looked down upon and are rarely used by most people aged 25-50. This group of people have two things in common - they typically know how to use a keyboard and trust its input, and they've all used the terrible voice commands from old phone systems in the past, which leads them not to trust voice. The lack of trust means few people use it, making it seem weird to do in public. This is getting better, and certainly more people are using voice commands now than they were five years ago. But it's still an extremely small percentage of users that are voice power users. Ask yourself this: how many people do you know that send text messages with voice? What percentage of your friend group is that? I know two people - one of them was a voice engineer who had to test voice input on phones constantly. Voice will get better, but it's going to take time. More people need to use it before we can take more risks, and we need to develop systems that know when they're being addressed before we can get rid of action words.
- hnha 12y agoSomething like this already happens all the time if programs steal focus. I have no idea.how often I've been happily typing only to have some dialog pop-up and accept my newline Enter as confirmation for whatever it wanted. Things are better now that I switched to Linux and enabled all the prevention but it still happens every now and then. Sometimes I end up writing my passwords in a thieving application
- nateparrott 12y agoIt's not just voice input that's severely neglected by UI designers—it's text-based interface in general. Consider his example of choosing "California" from a list of states—most people would find it way easier to just type "CA" than to scroll and hunt for "California" in a list—but how often do you see an interface that prods you to do this, or even makes apparent that it's possible? (In some web apps that use custom menus, it isn't!) Why, when I want to apply a 10px blur in Photoshop, can't I just type "blur 10 pixels" instead of digging through 2 nested menus and a modal? WIMP interfaces are great for _discovering_ what an app can do, but terrible at letting you do a specific thing fast. Things like spotlight and wolfram alpha are a step forward, but we're still light-years behind where we could be if we took text-based UI—or hybrid WIMP/text UI—seriously.
- ar7hur 12y agoDustin is right: a truly pleasant user experience would necessary involve continuous listening (without hotword à la OK Google), as well as leveraging the user context much more than what Siri or Google Now do today. See for instance the vision depicted in the movie Her [1]. Beyond Google or Apple, a few startups (like us at wit.ai) are hard at work designing building blocks to help developers solve these problems. [1] https://wit.ai/blog/2014/02/24/her-the-movie https://wit.ai/blog/2014/02/24/her-the-movie
- matthew-wegner 12y agoMy personal hunch is that Apple is going to heavily roll out physical/location context in the next few years. If my friend has a future Apple TV, I should be able to say "Hey, Siri"* and have Siri recognize that it's me and make my entire media library available for playback. This isn't as hard as it sounds, I don't think--Apple only has to search my friend's contacts' voices, I'm probably on my friend's wifi already, etc. * iOS 8 includes the ability to say "Hey, Siri" at any time when your iPhone is charging, but I've had crap luck with it during the betas (trying to use it in my car).
- dllthomas 12y agoI've given a small amount of thought to attempting a shell designed for spoken interaction, and also - relatedly - a scripting language designed to be spoken. I've not really gotten anywhere, though. This doesn't really sound like the same thing as either of those, but close enough I wanted to mention.
- angersock 12y agoSo, let's assume that the issue is context (based on discussion, that seems to be a major consensus point). Obvious solution is to allow more context to better tune understanding of voice, right? Allowing better context is pretty much predicated on constant audio surveillance. Is this worth it, truly?
- anigbrowl 12y agoWhen I highlight the browser address bar, I should be able to just say “The Economist” and have it automatically find the address in my favorites and go there. Would be nice, but not as good as 'Computer_name...anything new in the Economist?' 'There's a provocative analysis of the arms trade and some particularly egregious punning of the sort you claim to despise.' I have a marvelous scheme to stimulate market demand for such services, but this comment is insufficiently well-funded to contain it.
- mwcampbell 12y agoThis post is right about one thing: Speech recognition technology is good at context-specific recognition, i.e. with a small grammar, as in VoiceXML IVR applications (anyone else remember those?). This has been true on typical PC hardware since at least 2000, so it should be easy to run that kind of speech recognition locally on a smartphone. But, last time I did anything serious with that kind of speech recognition, it still required a push-to-talk button or the like. Maybe a trigger phrase would work now.