9 ms·
Thoughts on Voice Interfaces
- ocdtrekkie 6y agoThere's a lot of really nicely summarized points here. I think it's a good "makes you think about things" bit to read if you're going to work in this space.
- Ajedi32 6y agoIt's a really interesting writing style. - Feels more like an outline than an article or blog post - Covers all the essential points - Wastes no space on superfluous language - I think I like it
- sytelus 6y agoThis is just the evolution of thought process. This is bullet list phase which essentially just gathering data. Next phase should be to digest this data to reveal patterns, identify best practices and that article would be more interesting - worthy of a good essay. Next phase would be to synthesize new facts, bring new perspective and fix old issues. That article would be Worthy of a research paper.
- zokier 6y agoOne thing that is not mentioned is that voice is horrible for internationalization. One issue with voice is that the language of the UI is far more deeply entangled with the language of the content.
- fiblye 6y agoAnd it's definitely not future-proofed. Languages can change a lot in decades, from pronunciation to vocabulary to grammar. People think it's crazy that we still have people editing 50 year old COBOL programs today. Imagine how annoying it'd be to have to use a 50 year old computer that hasn't had "accent updates" and needing to emulate the speech patterns of your grandparents to be recognized.
- 082349872349872 6y agoOr spatially-proofed: https://www.youtube.com/watch?v=TqAu-DDlINs https://www.youtube.com/watch?v=TqAu-DDlINs "Please? Pathetic." (Autocorrected SMS was unpopular in my country when it was first introduced, because it corrected to german german spellings, and ever since the mid-twentieth, we've been proud of regional orthographies. I imagine francophones who use "SMS language" also have to turn off autocorrect.)
- sradman 6y ago> Many systems use a “wakeword” or “keyword spotting” to start the interaction. What if we used keyword spotting to determine the end as well? “Please” might be a good choice. It’s like the Enter key. I hate saying "Alexa, dismiss" for Reminder notifications rather than accepting the natural conversational "OK", "Thanks", "Got it", or "OK, thanks, I got it"... Polite language sometimes conveys information: "please" signals an expectation of action, "thank you" can act as an acknowledgement or mark the end of a conversation, etc.
- deleted 6y ago[deleted]
- jldugger 6y agoMe: Alexa, text my spouse please Alexa: What would you like to say? Me: Thank you for reminding me to pack my lunch, the cafeteria options were terrible today! Alexa: what would you like to say?
- echelon 6y ago> Many systems use a “wakeword” or “keyword spotting” to start the interaction. What if we used keyword spotting to determine the end as well? “Please” might be a good choice. It’s like the Enter key. This would solve so many problems. Sometimes I like to think a bit about what level to set the lights at, but by that time the machine thinks I'm done with my request. We need richer interaction models, and this is an area where it'd be nice if we could focus on programmers first to find the appropriate mechanisms. I think modal systems that set contexts would also be amazing. "Robot, let's adjust the lights" Which could set the system into dealing only with lighting-related requests.
- falcolas 6y agoA combination of Star Trek and CB Radio gives us some pretty good starting points: Computer, set lights at... 50%. Over.
- nitrogen 6y agoIsn't it significantly faster and more accurate to just turn a knob or move a slider to the middle? 500ms and no conversational interruption vs. 5-10 seconds where everyone has to be quiet.
- toast0 6y agoIf you're near the knob, and you have a clean hand, and a free hand, and your motor control is fine, and your lights are wired to switches how you want to control them (some people want individually controlled bulbs, but that's a lot of switches and knobs). BTW, always on the lookout for an offline only voice activated clock with multiple timers. wifi for NTP only is also acceptable; no cloud allowed. 98% of what we use the voice assistants for is task timers; it's nice to have those voice activated so everyone knows they were set.
- pmx 6y agoThere seem to be some efforts to make Mycroft run without an internet connection here https://community.mycroft.ai/t/the-mycroft-personal-server-starting-the-conversation/4691 https://community.mycroft.ai/t/the-mycroft-personal-server-s...
- k__ 6y agoI think, all this AI is overkill. If those things would be more active they would work much better.
- AriaMinaei 6y agoHow about a high-bandwidth vocal signal language in place of natural language? Here is an example: https://youtu.be/sG3PWet8fDk?t=4545 https://youtu.be/sG3PWet8fDk?t=4545
- baddox 6y agoI feel like you'd lose most of the advantage of a voice interface if you had to learn a new (and presumably difficult) high-bandwidth vocal signal language. It might still be useful for some niche scenarios where it's very important to have your hands free or to operate over an audio channel, but the real notable advantage of a voice interface is that most humans already use speech as their primary method of communication with other humans.
- AriaMinaei 6y agoThe learning curve would certainly be a disadvantage. But just like learning to play an instrument, the payoff could be worth it.
- floatrock 6y agoI've heard a trick that acrobatic flight squads like the Blue Angels use when coming into really tight formations is each pilot hums at a specific frequency/tone. By varying their pitch up or down, they can signal to the other pilots what they're about to do, kinda like an audio turn signal. And because each pilot is humming at a different frequency, everyone in the squad can be communicating at the same time -- in the same way you can pick out different instruments in a song, our ears can separate different simultaneous humming at different frequencies. Not sure this is what the author meant in the first section when he wrote: > I suspect you can push the user into a conversational skeuomorphism if you think that’s best, and the user will play along, but it’s no more right than another metaphor. It’s a question of quality of interaction, not ease or familiarity. ...but it's an interesting example of non-conversational multiplexed voice communication.
- ghaff 6y agoI think what he's saying is that you can push people towards "natural" conversation but if it doesn't really work well, people will get frustrated. Which is sort of the problem with voice assistants today for the most part. You really do have to get the right wizard incantations a lot of the time.
- killion 6y agoSo many great points. I'm very happy to hear that someone is thinking of the UX. I only wished they worked on Siri. The only addition I can think of is during the wake word section. I'd like to remove the wake word requirement if I'm in the middle of an interaction. For example, sending a text on Siri. "Hey Siri text my wife. Tell her I'm on my way" Siri shows me the text and then I have to again say "Hey Siri send it" when just "Send it" should do.
- jagged-chisel 6y agoTrying your example starts an interaction with Siri for me. Siri verifies the message content, asks if I’m ready to send it, and listens for my reply. Using AirPods Pro is supposed to allow somewhat reduced vocal utterances to Siri, but I haven’t spent time learning that.
- lazyjeff 6y agoAgreed. This article is so loaded with not-so-obvious observations that can be used as a source of ideas for mini research projects. It's a goldmine for things to try. Kudos to the author for sharing their thoughts on this space.
- cookingrobot 6y agoI fed the beginning of this post into GPT-3 and it came up with these additional thoughts. I’m extremely interested in how voice interfaces can be improved with a language model like this. Unedited GPT-3 output: “““ I think this problem is especially bad when you have to use a specific vocabulary, and when you have to speak in a specific order. It’s not just the laggy UI, it’s the idea that your vocabulary and phrasing is not flexible. The laggy UI is an artifact of that lack of flexibility. I don’t think it’s hard to understand how frustrating that can be. A lot of people in the voice community are concerned with “flow”. This is the idea that the conversation should just flow, that we shouldn’t have to interrupt it to do work. Flow can be achieved in other interfaces by turning off prompts, and putting in defaults. But it’s hard to do in voice. First, if you have to wait too long, then you are in trouble. Second, you can’t do defaults, because the only way to get the answer is to wait for the user to speak. I think that is true. But there are ways to achieve a kind of flow, by turning off prompts and being very responsive. We don’t have to show you all the information on the screen at once. We can use highlights to let you know the computer is paying attention, or make use of non-visual modalities to let you know something happened. If we show you something on the screen, it’s because we are trying to convey something important, and you’ll be able to act on it, but we can’t expect you to act on everything. Even with this in mind, flow is a design principle, not a technical requirement. I don’t think we should design systems that are trying to achieve flow, because it’s very hard to do. I do think we should design systems that are responsive and forgiving. Voice systems have a unique capability: they can communicate with you without being able to see you. This is the key to being able to make an AI feel more like a person. If I could have a computer read my mind, that would be much more powerful than if it just read my words. This is the reason that voice is so compelling, and why it’s so interesting to implement it. Flow is a principle of interaction design, and it’s about putting in a lot of work to make a system seem less work. Voice does not make interaction feel less work. In fact, voice can be the opposite. It is work. Voice requires us to work harder, and because it is so personal, we feel that more. Voice is a more intimate interface. You are using your voice to speak to a computer. There is a large class of problems which are better solved with a screen. I believe this to be ”””
- IshKebab 6y agoHow has nobody connected GPT-3 up to a voice interface yet? It wouldn't reveal anything new but it I think the level of conversation GPT-3 is capable of combined with actually talking to an AI would be fairly mind blowing.
- jedberg 6y ago> Some people think it is important that we not abuse assistants. They believe abuse will make us cold or abusive to each other. I do not agree. And then he provides four bullet points explaining exactly why we should not abuse assistants...
- pseudalopex 6y agoHe says we have the opposite problem. People feel judged by their talking computers and feel sorry for tools. He says we should make systems we don’t anthropomorphize.
- nshm 6y agoThe most important point Firefox people forget: it is better to be a good browser than bad Alexa.
- deleted 6y ago[deleted]
- evrydayhustling 6y agoThere are so many great thoughts in here... Only halfway through but one of the most substantive UI pieces (or any piece) I've seen in a bit. Lots of this is relevant beyond voice, to then widening range of immersive interfaces around us (text bots, dynamic alerts and recommendations, etc.) One thing on my mind with voice is that humans generate useful shorthand very dynamically. It would be great if I could conversationally set up more context with Alexa about tasks I do often or music I like, and have that improve efficiency and accuracy when I repeat. Instead, I experience the opposite: memorized commands often stop working because the outside world has changed; for example a song I like getting shadowed by a crappy (but newer) cover.
- nullc 6y agoA little offtopic, but am I the only person that is driven absolutely batty by stuff like google maps talking over conversations out of nowhere and without warning (usually telling me about something of no value)? I really wish these text-to-speech interfaces would begin notivation with a unobtrusive ding, wait a moment so you can stop talking... or even better, listen and slightly delay their notice so they're not talking over you if at all possible.
- gruez 6y ago>or even better, listen and slightly delay their notice so they're not talking over you if at all possible. with ios adding an indicator to let the user know whether an app is listening in or not, I'm sure this is going to cause a lot of pushback from users in the form of "omg google is eavesdropping on everything we say!!!".
- programmarchy 6y agoYes, especially when listening to a podcast in the car. "And so now begins the most interesting twi--" "Get ready to make a left turn up ahead" "--and it was all because--" "Left turn in 100 meters" "--but what really--" "Turn left" "--and that would change things forever."
- woadwarrior01 6y agoNot so with CarPlay. I’ve noticed that while using the Podcasts app and Apple Maps with CarPlay, it pauses the podcast while the maps app is speaking and then resumes it.
- nullspace 6y agoI think it depends on the specific app. Does what you’re saying in Pocketcasts, but the Economist app does what parent is saying.
- deleted 6y ago[deleted]
- 6y ago
- dafoex 6y agoNow, my sample size is small, but I've noticed that people react to voice interfaces with what I can only think to call "dumb foreigner syndrome" where they actively slow down and simplify their speech as if they were talking to someone that barely understood the language. I think this comes from familiarity with bad voice interfaces, but maybe we should indulge this expectation of basic but explicit commands, rather than rich but imprecise language. Dare I say it, but maybe we should look to the likes of Zork for inspiration for voice input.
- macrolime 6y agoI always do this. It's because there are only bad voice interfaces. Siri,Alexa etc, it's utter crap.
- cxr 6y ago> Dare I say it, but maybe we should look to the likes of Zork for inspiration for voice input. On that note, there's nothing intrinsic here, really, to why these need to be voice interfaces. Based on watching the video[1] and my experience using voice interfaces, if my hands are already on the keyboard, I'd wager I can type some of these things as quickly as if I'd spoken them and waited for the agent to keep up (even assuming they can). I'm also surprised that the Awesomebar-/Sublime Text-/VSCode-style command palette hasn't by now become a ubiquitous feature of user interfaces. I'm really surprised they haven't replaced for many use cases the combo of Bash running in a terminal emulator, given how well-known a problem it is to have to remember the args to infrequently used commands. And I'd rather type "it" (as in in delete it) than to ever type "$_" even once—let alone ask someone with a straight face to remember that this is the way to do things when you're talking to the computer. 1. https://www.youtube.com/watch?v=3sqKsfj8WRE https://www.youtube.com/watch?v=3sqKsfj8WRE
- dafoex 6y agoMy comment about Zork comes from Don Gentner and Jakob Nielson's "The Anti-Mac Interface", which mentioned a simple search bar that could take more-or-less natural language instructions such as "make a new document called outlines" and the computer would make a new word processor document with that file name and maybe automatically title it for you. It sounds like it may be a similar thing to your command palette, or maybe something like the Unity HUD or those Rofi launchers I keep seeing i3 users showing off. Come to think of it, I usually type into my Google Assistant app on my phone, rather than talk to them with voice, so there's no need to make these voice exclusive.
- Nimitz14 6y agoGreat blogpost, thanks for sharing. I'm hoping the next 5 years will lead to enough improvement that accuracy levels will get high enough that users will feel more comfortable using voice interfaces. We'll see. It does seem to me like as long as it can only be used for commands it will never take off. But being able to do conversations consistently is a very hard problem.
- lkrubner 6y agoA few of my own thoughts from when I was working on this full-time: "Dialogue designers replace graphic designers when creating voice interfaces" http://www.smashcompany.com/technology/script-designers-replace-graphic-designers-when-creating-voice-interfaces http://www.smashcompany.com/technology/script-designers-repl...
- david_draco 6y agoMaking voice input like talking to a person, is like trying to make a GUI similar to an oven interface. Don't try to use full sentences to command a computer. People are happy to learn a few words if accuracy increases to >99% and it is more convenient. Also, incorporate humming in tones.
- earthboundkid 6y agoI’m surprised neither Google nor Apple have taken voice reading of webpages out of the accessibility ghetto and made them full fledged interfaces yet. Why can’t I walk the dog and say “hey dingus, read Hacker News” and get a decent experience without having to look at my phone or put it in a voice UI only mode.
- artpi 6y agoDiscovery is IMO the biggest reason why voice interfaces will never take off. Mapping the space of possible solutions is delegated to your brain. You are responsible for remembering what's possible, what you can do, and what you should say to make that happen. The majority of the time you will fail, because you forgot what the magic spell actually was. Visual interface has an unobtrusive way of showing you what you can do. More importantly - it can show you multliple options at once. Voice feedback can only tell you one thing at a time and very slowly. Voice assistants are called that for a reason - core assumption is that you share your intent, and assistant is smart enough to make that happen. But that does not work with people either! Nobody can read your mind (yet) - that is why project requirements and management exists. I would love to tell my team one sentence of what I want done, but I have to review that at some point. I am not using voice assistant to do human things badly. I want it to do computer things well.
- throwaway_pdp09 6y agoLooks like a good article. I'll look more closely in a bit. Re. your comment about discovery, I use an old version of a voice recognition software (dragon). From the help: "Say "What Can I Say?" to see a list of sample commands you can say in the current application you are using" So not a problem, at least in voice transcription software. From the article criticising recognition loss in brevity, which is very true: > “Paste” is almost never detected correctly. Yup, so dragon simply expands the phrase to "paste that". It uses such redundancy to fix the problem, and because words are cheap (if you have no voice disability) this IME works well. I have to say, having used dragon a long time now, I don't see much value in voice command interfaces primarily but it totally shines in plain old text transcription.
- codebeaker 6y agoThis comment probably doesn't add to the conversation, so I apologise in advance. We make extensive use of Sonos and Amazon Echo products in our household and have two young children (3 and 5) the older one doesn't like issuing instructions for the Echo devices because he pauses occasionally and it somewhat aggressively "SORRY, I CAN'T HELP RIGHT NOW". A bigger problem we've noticed is that things "drift". We've got muscle memory of asking "Alexa, play classical lullabies" or "Echo, play the skeleton dance song". Occasionally though these commands will be "hijacked" by some new content in Spotify or somewhere else. "Alexa, play music for children" used to play nice kids songs, now it plays some murder-metal album called (one assumes ironically) "music for children", prior to that match, it seems like it was a search, or a playlist? And there seems to be no way to deny-list music like this. It's driving us off of Spotify in our household, "skeleton dance" plays either screaming metal artists or the cutesey kids song with about 50% accuracy, it seems. FWIW I had the same problem with Amazon FreeTime where for 5 bucks a month you get access to 10,000 kids shows and games, which is about 9950 more than I had time to vet to check if they were appropriate for my kids. Please, hold your criticism if you want to berate me for raising kids in a household with voice assistants and tablets, I am trying to balance exposing them to useful technology without exposing them to the underworld of utter shite that is to be found beneath those interfaces.