5 ms·

A Natural Language User Interface is just a User Interface

88 points by syllogism 10y ago

nerdponx 10y ago
I'm glad someone wrote this article. "Chat-based" interaction is a lot like phone systems that ask you to describe what you're calling about, instead of giving a clear set of options. In my case it least, it usually ends in frustration, and falling back to the list of presets anyway. If I'm in a noisy area, or reception isn't perfect and the robot at the other end can't understand what I'm saying, then it's downright hopeless.
- Mikhail_Edoshin 10y ago
  This was exactly my experience when I phoned my ISP and was met by an obviously recorded message that asked me what I want and then, apparently, switched to listening. I had no idea what to say to it! So I waited until it returned to the usual "wait to switch to an operator". Only a few calls later I learned from the human operator that I'm supposed to say something like "technical support". (At the same time I have no difficulty and even prefer to call a local taxi using a voice menu based on my usual routes. These guys jump straight to business: "if your current address is X, press 0", "if you're going to Y, press 0", and then confirm; takes half a minute.)
wrong_variable 10y ago
I think the main 'magic' of NLUI is not the mapping. This person is projecting the model of how GUI works to NLUI. A better way to think about NLUI is having another person have access to the real GUI - and you are merely commanding the person to achieve certain goals. The thing about advanced NLUI is that it can convert really vague context sensitive commands to highly structured set of sub-routines that programmers will never be able to do easily. 'what should I major in ?' 'which university should I apply to' 'what headphones is best ?' ....
- syllogism 10y ago
  I find your examples sort of surprising? These don't seem so structured. Imagine a GUI that had three buttons: Recommend University | Recommend Major | Recommend Headphones Now...this set of capabilities is sort of an odd mix. But your application is going to be amazing if you can deliver on what your UI is promising here. If you can help someone make a better decision on their degree, that's life-changing. The UI that triggers that functionality seems quite beside the point.
Kaotique 10y ago
The power of the chat interface is that you can include multiple bots and features in the same UI. No need for separate screens with a search field, filters and buttons. It's all in the same chat.
- dyarosla 10y ago
  You're assuming that typing/talking to a bot is the easiest way to get things done. That's simply not true. We have custom UIs to interface with different domains because different domains work better/more efficiently with different UIs. The following post has an analysis on this, with many great examples in the real world today http://dangrover.com/blog/2016/04/20/bots-wont-replace-apps.html http://dangrover.com/blog/2016/04/20/bots-wont-replace-apps....
  - peter_l_downs 10y ago
    The linked article is excellent -- really appreciated the history lesson. Thank you for the pointer.
Aardwolf 10y ago
I'm waiting for the day where I can verbally say to my computer: "No, please do not snap back the scrollbar when I move my mouse pointer farther away, keep scrolling!" or "No, bad computer! Do not select the entire URL if I single click, only when I double click!" or "It's a PDF! Don't give me a download box, render the PDF in the browser now!" and the computer will understand what I mean, and update the GUI to work as I said.
- gumby 10y ago
  RMS has this system: control-P! control-P foo control-s bar return Works for him.
juandazapata 10y ago
The way I see it, this hype is a bridge to controlling software with voice. We're in a transition period and soon(?) our voice will be the UI. Remember that the best UI is no UI at all.
deleted 10y ago
[deleted]
bravura 10y ago
Here's a talk my colleague and I gave at O'Reilly Strata about natural language UIs: https://www.youtube.com/watch?v=lnV2JnNBM1I https://www.youtube.com/watch?v=lnV2JnNBM1I We focus on NLUIs for the purposes of data querying. In many cases, natural language is concise and allows you to elide implied constraints. For example, "Opportunities where I'm on the account team" The SQL would be: SELECT * FROM OPPORTUNITY WHERE EXISTS (SELECT 1 FROM account_team_member atm JOIN opportunity ON opportunity.account_id = atm.account_id WHERE atm.user = me) There's a correlated subquery in there. Even if you are a SQL expert, your time to query formulation is far higher if you use SQL than English. Drag and drop UIs are just a thin veneer over SQL, so a drag and drop UI requires understanding the underlying data model and join structure. My argument here is that there are some key places where NLUIs really shine, especially when they are scrupulously combined with more traditional UIs.
- nitwit005 10y ago
  My previous team studied allowing similar queries, and we didn't see anything so easy to interpret into a query. In testing, users typed in a lot of strings that couldn't be interpreted (by machine or human), or which were ambiguous. It may be that people have gotten used to Google/Bing accepting crap queries and still showing a decent rezult. But, I suspect the main problem is they couldn't explain what they wanted to a human, at least not without considerable conversation.
- syllogism 10y ago
  This is indeed very nice :). I did my thesis on CCG, so this is all very dear to my heart. The paper I'm most proud of is CCG work too.[1] I have some unpublished follow-ups on light verb constructions I'd love to get released one day. Btw, upshotdata.com returns 403 --- probably this isn't news to you, but just in case. [1] http://www.aclweb.org/anthology/P10-1022 http://www.aclweb.org/anthology/P10-1022
- stcredzero 10y ago
  Your example has some contextual information contained in the overall structure of the sentence. There's also information in the text which wouldn't be contained in a vocal query. Your example might be answered by a query with the following link: http://mars.nasa.gov/mer/mission/tm-opportunity/opportunity-sol4408.html http://mars.nasa.gov/mer/mission/tm-opportunity/opportunity-... Followed by the system registering that you are "on the account team." I do agree that NLUIs can shine where there are constrained/specific interaction contexts.
niftich 10y ago
The bot hype right now isn't about average UIs being replaced with text prompts of "What would you like to do today?", it's about large walled-garden ecosystems (Google, FB, Amazon, Apple, MS, Kik, etc.) funneling web searches and services through their own systems. The article's most crucial point is that a natural language UI is just another interface to the limited set of functionality the application is capable of. But to the current big players in these game, these limitations don't apply: if the parsed commands fall through all of their many layers of specialty logic, they can always fall back to a generic web search. By doing so, they will have provided the user with an interface that tries to understand what they want, while being no worse than if the user had typed the same thing into a search bar. And all the while, they have routed the user's intent through their AI, enabling better personalized recommendations, and tracking of click-through rates.
ChoHag 10y ago
I'm stumped. Where have I seen before one of these Interfaces where users type a Line with a Command?

88 points by syllogism 10y ago

I'm glad someone wrote this article. "Chat-based" interaction is a lot like phone systems that ask you to describe what you're calling about, instead of giving a clear set of options. In my case it least, it usually ends in frustration, and falling back to the list of presets anyway. If I'm in a noisy area, or reception isn't perfect and the robot at the other end can't understand what I'm saying, then it's downright hopeless.

Mikhail_Edoshin 10y ago
This was exactly my experience when I phoned my ISP and was met by an obviously recorded message that asked me what I want and then, apparently, switched to listening. I had no idea what to say to it! So I waited until it returned to the usual "wait to switch to an operator". Only a few calls later I learned from the human operator that I'm supposed to say something like "technical support". (At the same time I have no difficulty and even prefer to call a local taxi using a voice menu based on my usual routes. These guys jump straight to business: "if your current address is X, press 0", "if you're going to Y, press 0", and then confirm; takes half a minute.)

wrong_variable 10y ago

I think the main 'magic' of NLUI is not the mapping. This person is projecting the model of how GUI works to NLUI. A better way to think about NLUI is having another person have access to the real GUI - and you are merely commanding the person to achieve certain goals. The thing about advanced NLUI is that it can convert really vague context sensitive commands to highly structured set of sub-routines that programmers will never be able to do easily. 'what should I major in ?' 'which university should I apply to' 'what headphones is best ?' ....

syllogism 10y ago
I find your examples sort of surprising? These don't seem so structured. Imagine a GUI that had three buttons: Recommend University | Recommend Major | Recommend Headphones Now...this set of capabilities is sort of an odd mix. But your application is going to be amazing if you can deliver on what your UI is promising here. If you can help someone make a better decision on their degree, that's life-changing. The UI that triggers that functionality seems quite beside the point.

Kaotique 10y ago

The power of the chat interface is that you can include multiple bots and features in the same UI. No need for separate screens with a search field, filters and buttons. It's all in the same chat.

dyarosla 10y ago
You're assuming that typing/talking to a bot is the easiest way to get things done. That's simply not true. We have custom UIs to interface with different domains because different domains work better/more efficiently with different UIs. The following post has an analysis on this, with many great examples in the real world today http://dangrover.com/blog/2016/04/20/bots-wont-replace-apps.html http://dangrover.com/blog/2016/04/20/bots-wont-replace-apps....
- peter_l_downs 10y ago
  The linked article is excellent -- really appreciated the history lesson. Thank you for the pointer.

Aardwolf 10y ago

I'm waiting for the day where I can verbally say to my computer: "No, please do not snap back the scrollbar when I move my mouse pointer farther away, keep scrolling!" or "No, bad computer! Do not select the entire URL if I single click, only when I double click!" or "It's a PDF! Don't give me a download box, render the PDF in the browser now!" and the computer will understand what I mean, and update the GUI to work as I said.

gumby 10y ago
RMS has this system: control-P! control-P foo control-s bar return Works for him.

juandazapata 10y ago

The way I see it, this hype is a bridge to controlling software with voice. We're in a transition period and soon(?) our voice will be the UI. Remember that the best UI is no UI at all.

deleted 10y ago

[deleted]

bravura 10y ago

Here's a talk my colleague and I gave at O'Reilly Strata about natural language UIs: https://www.youtube.com/watch?v=lnV2JnNBM1I https://www.youtube.com/watch?v=lnV2JnNBM1I We focus on NLUIs for the purposes of data querying. In many cases, natural language is concise and allows you to elide implied constraints. For example, "Opportunities where I'm on the account team" The SQL would be: SELECT * FROM OPPORTUNITY WHERE EXISTS (SELECT 1 FROM account_team_member atm JOIN opportunity ON opportunity.account_id = atm.account_id WHERE atm.user = me) There's a correlated subquery in there. Even if you are a SQL expert, your time to query formulation is far higher if you use SQL than English. Drag and drop UIs are just a thin veneer over SQL, so a drag and drop UI requires understanding the underlying data model and join structure. My argument here is that there are some key places where NLUIs really shine, especially when they are scrupulously combined with more traditional UIs.

nitwit005 10y ago
My previous team studied allowing similar queries, and we didn't see anything so easy to interpret into a query. In testing, users typed in a lot of strings that couldn't be interpreted (by machine or human), or which were ambiguous. It may be that people have gotten used to Google/Bing accepting crap queries and still showing a decent rezult. But, I suspect the main problem is they couldn't explain what they wanted to a human, at least not without considerable conversation.
syllogism 10y ago
This is indeed very nice :). I did my thesis on CCG, so this is all very dear to my heart. The paper I'm most proud of is CCG work too.[1] I have some unpublished follow-ups on light verb constructions I'd love to get released one day. Btw, upshotdata.com returns 403 --- probably this isn't news to you, but just in case. [1] http://www.aclweb.org/anthology/P10-1022 http://www.aclweb.org/anthology/P10-1022
stcredzero 10y ago
Your example has some contextual information contained in the overall structure of the sentence. There's also information in the text which wouldn't be contained in a vocal query. Your example might be answered by a query with the following link: http://mars.nasa.gov/mer/mission/tm-opportunity/opportunity-sol4408.html http://mars.nasa.gov/mer/mission/tm-opportunity/opportunity-... Followed by the system registering that you are "on the account team." I do agree that NLUIs can shine where there are constrained/specific interaction contexts.

niftich 10y ago

The bot hype right now isn't about average UIs being replaced with text prompts of "What would you like to do today?", it's about large walled-garden ecosystems (Google, FB, Amazon, Apple, MS, Kik, etc.) funneling web searches and services through their own systems. The article's most crucial point is that a natural language UI is just another interface to the limited set of functionality the application is capable of. But to the current big players in these game, these limitations don't apply: if the parsed commands fall through all of their many layers of specialty logic, they can always fall back to a generic web search. By doing so, they will have provided the user with an interface that tries to understand what they want, while being no worse than if the user had typed the same thing into a search bar. And all the while, they have routed the user's intent through their AI, enabling better personalized recommendations, and tracking of click-through rates.

ChoHag 10y ago

I'm stumped. Where have I seen before one of these Interfaces where users type a Line with a Command?