3 ms·

A UI is basically a trade tongue. We don't speak machine and machines don't speak human. A UI is a least common denominator that everyone can learn quickly. Bu

by netcan 11y ago

A UI is basically a trade tongue. We don't speak machine and machines don't speak human. A UI is a least common denominator that everyone can learn quickly. But it's nowhere near native quality. You can't get much more than simple one liners in trade talk. Good enough for negotiating a chest of cloves, but you'd be hard pressed to explain the difference between marxist-leninist and labour socialist perspectives on international commerce in a trade tongue. When people hear "voice interface" what they think is a machine that can understand human so we can stop speaking goddamned pidgin all day. That is essentially a turing test, or some subspecies thereof. I'm somewhat skeptical about this kind of thing. I'm confident that mechanical issues like translating sounds to sentences can be solved (the voice part). The translation of human to machine code… That's a different challenge. It's an interesting case though, to have out in the wild. Siri and whatnot. An application (as opposed to an average starship computer) has a limited raison d'être, therefor a much tighter search space. Meanwhile, we have been making consistent progress on turning our clunky pidgins into nice creoles, more elegant and powerful. Machines can understand higher level programming languages that people can get more fluent in. Average humans are generally becoming more computer literate (or fluent in UI, to keep my analogy straight). UIs are getting more intuitive, which basically means closer to human language. The common ground is expanding from just common ground and a wider language is being developed that accommodates us both better. The question to answer is what part of that common ground is voice is open, I guess. But, I would wager that the common ground we find will not be human sounding anytime soon. It will be a UI metaphor, like the desktop. We will need to learn it.

wanderfowl 11y ago
This is a really well put explanation of why the natural language processing side is hard, but it assumes that we can even get letters and words into the system. I'm a linguist who works on speech perception, and I'm increasingly convinced that true speech-to-text, that can stand in for a human transcriptionist, is an AI-complete problem. We can get close, but you'll always need context and understanding of the world to accurately get things like "I'm going to take a walk in the park, then I'm going to take a wok from the Chinese Restaurant". Siri and whatnot are amazing, and there's room for improvement, especially on the NLP end, but these issues combined with how damned hard speech is mean that Jarvis will be a keyboard away for years to come.
bane 11y ago
That's a fantastic analogy. I had a worrying discussion the other day with somebody about user interfaces. I was asserting that UIs are more than just CLIs and GUIs, more than VUIs (Voice User Interface)...UIs extend to virtually everything we do to interact with a computer. A programming language is a kind of UI, an API is a kind of UI and so on. I was asserting that when we think of the entire interaction surface of a computer, from bare metal on up, as an interface (with us as the users), then we need to make sure we design those interfaces to work well as an interaction medium because it enables "better" interaction with machines (on virtually every vector).
Lambdanaut 11y ago
"The" native quality is complete brain-computer merging. Brain-computer direct communication is close but not quite there. It's not 100% until your thoughts are completely merged with the computer's. Your thought can't be called your own anymore, and it can't be called the computer's thought. At this point, you and the machine have become one, and latency has been reduced to zero.
cmaury 11y ago
Absolutely right. We refer to this as the Scarlett Johanssonn problem. Any system that is smart enough to understand what we mean by the Scarlett Johanssonn problem, and the reference to Artificially Intelligent virtual assistants, would itself need to be a strong AI. We've already solved the problem of translating human to machine code for graphical user interfaces. It's the desktop with the keyboard and mouse. It's the app and the touchscreen. You're exactly right that the question now is how can we do the same translation for voice-based systems. So far the approach has been Virtual assistants, which is really hard, see again the Scarlett Johanssonn problem. We've been working to find the right design/interaction metaphor, and the app metaphor seems to work really well. Not only is the search space limited, like you mentioned, but users have a more intuitive since of how to interact with the app. When you open a shopping app, you know that you can likely search for something to buy ("I'm looking for a new pair of shoes"). When you interact with a general virtual assistant, that interaction is less clear. The app model also works for scaling conversational interaction beyond a single context. Individual apps can support their own language for interaction which best suits their function. This of course necessitates design standards, just like we have for GUIs. The limitations for fully voice-based applications aren't technical at this point. They are design problems with known solutions. The break out for voice will happen when the right form factor comes about with the right UI design.
- Isamu 11y ago
  >The break out for voice will happen when the right form factor comes about with the right UI design. Chris Maury? If so, hoping to see your progress on this with Conversant Labs in the coming year or so. After I read the article I was wondering if I'd see you on HN. Also my daughter (who also has Stargardt's) found it very inspiring to chat with you at the accessibility meetup.
  - cmaury 11y ago
    You caught me :). It was great meeting you daughter! She seems to be doing great and taking things on with the right attitude. Feel free to email me anytime, if you ever need anything: Chris@Conversantlabs.com. Happy to talk through things in a more private setting than the comments section of HN.

by netcan 11y ago

wanderfowl 11y ago

This is a really well put explanation of why the natural language processing side is hard, but it assumes that we can even get letters and words into the system. I'm a linguist who works on speech perception, and I'm increasingly convinced that true speech-to-text, that can stand in for a human transcriptionist, is an AI-complete problem. We can get close, but you'll always need context and understanding of the world to accurately get things like "I'm going to take a walk in the park, then I'm going to take a wok from the Chinese Restaurant". Siri and whatnot are amazing, and there's room for improvement, especially on the NLP end, but these issues combined with how damned hard speech is mean that Jarvis will be a keyboard away for years to come.

bane 11y ago

That's a fantastic analogy. I had a worrying discussion the other day with somebody about user interfaces. I was asserting that UIs are more than just CLIs and GUIs, more than VUIs (Voice User Interface)...UIs extend to virtually everything we do to interact with a computer. A programming language is a kind of UI, an API is a kind of UI and so on. I was asserting that when we think of the entire interaction surface of a computer, from bare metal on up, as an interface (with us as the users), then we need to make sure we design those interfaces to work well as an interaction medium because it enables "better" interaction with machines (on virtually every vector).

Lambdanaut 11y ago

"The" native quality is complete brain-computer merging. Brain-computer direct communication is close but not quite there. It's not 100% until your thoughts are completely merged with the computer's. Your thought can't be called your own anymore, and it can't be called the computer's thought. At this point, you and the machine have become one, and latency has been reduced to zero.

cmaury 11y ago

Absolutely right. We refer to this as the Scarlett Johanssonn problem. Any system that is smart enough to understand what we mean by the Scarlett Johanssonn problem, and the reference to Artificially Intelligent virtual assistants, would itself need to be a strong AI. We've already solved the problem of translating human to machine code for graphical user interfaces. It's the desktop with the keyboard and mouse. It's the app and the touchscreen. You're exactly right that the question now is how can we do the same translation for voice-based systems. So far the approach has been Virtual assistants, which is really hard, see again the Scarlett Johanssonn problem. We've been working to find the right design/interaction metaphor, and the app metaphor seems to work really well. Not only is the search space limited, like you mentioned, but users have a more intuitive since of how to interact with the app. When you open a shopping app, you know that you can likely search for something to buy ("I'm looking for a new pair of shoes"). When you interact with a general virtual assistant, that interaction is less clear. The app model also works for scaling conversational interaction beyond a single context. Individual apps can support their own language for interaction which best suits their function. This of course necessitates design standards, just like we have for GUIs. The limitations for fully voice-based applications aren't technical at this point. They are design problems with known solutions. The break out for voice will happen when the right form factor comes about with the right UI design.

Isamu 11y ago
>The break out for voice will happen when the right form factor comes about with the right UI design. Chris Maury? If so, hoping to see your progress on this with Conversant Labs in the coming year or so. After I read the article I was wondering if I'd see you on HN. Also my daughter (who also has Stargardt's) found it very inspiring to chat with you at the accessibility meetup.
- cmaury 11y ago
  You caught me :). It was great meeting you daughter! She seems to be doing great and taking things on with the right attitude. Feel free to email me anytime, if you ever need anything: Chris@Conversantlabs.com. Happy to talk through things in a more private setting than the comments section of HN.