5 ms·

Yet, no one is tackling the AI UI problem. Why is that? I can't believe people are still thinking natural language is the way to go...

ethbro 9y ago
Natural language is always the endgame. Because natural language either removes the physical input system or frees up screen real estate.
- kobeya 9y ago
  Natural language can be incredibly imprecise though. I expect the ideal UI for AI will meet somewhere in the middle with something that combines the best of programming languages with the specificity and ease of use of natural language -- think "strongly typed natural language". I'd rather my command type-error than the machine go off and do what I say, not what I meant.
  - ethbro 9y ago
    Step 1 was recognizing speech to text accurately. Step 2 is AI that is globally context- and subtext-aware. I expect Step 3 to be AI that is personally situation-, context-, and subtext-aware (IoT & personal info storage-enabled). And I think that's drastically going to improve accuracy. I can see some standardization of input (in the same way that GUIs evolved), but I think it will probably take the form of grammar structure & error behavior more than strongly typing questions.
    - dkersten 9y ago
      That will solve the AI understanding you, but still won't make me more precise and clear in my instructions. Yes, the AI can hopefully infer the rest, but even then, I can currently type faster than I can speak. Will the AI be able to infer enough of what I want that I can say substantially less and have the AI still get it right? At that stage, why not go all the way and remove me from the equation completely?
      - ethbro 9y ago
        On the typing angle, I'd argue the HN vs mass market point. Typing is still a skill we aren't born with. As much as it's now assumed in 1st world core demographics, there are a huge number of people for whom it isn't faster. And if you've ever watched a grandparent try and use Windows, there are a lot of other metaphors they miss too. I'm not saying NL recognition is going to be easy, but I am saying that getting there will be far more valuable than maybe the HN crowd assumes from self-reflection.
        dkersten 9y ago
        It depends on the use case. If its a typical Google Home-style "turn on the lights", "look up this thing", "do my accounts" then I agree (although the AI may need to do a lot of prompting since people (at least me) often don't know what we really want). For less casual uses, I'm not sure. I guess there's a spectrum.
        ethbro 9y ago
        That's a good point on the prompting, and something I don't believe a lot of current systems do? E.g. "Do X" "Would you like me to X1 or X2?" I'd guess the complexity usually isn't there as (afaik) current systems tend to be single action endpoint rather than knowledge parsing the request.
        dkersten 9y ago
        Yeah I think you're right that current systems don't have the complexity yet. It's either simple tasks (turn on the lights, play music), search (find X for me) or menus. As complexity of tasks increases, I think it'll become more important.
- ragebol 9y ago
  Yet, I don't like to use e.g. Google Now when I'm not alone. Dictating a personal text message to my phone in public is not what I want.
  - vthallam 9y ago
    Yep. that's why they allow both chat and voice as input to the Google assistant. Voice is only one way of accessing consumer AI
  - ethbro 9y ago
    Granted, but I'd expect subvocalization with predictive text or something else to infill where necessary. Stronger AI can substitute for a lot of "magic."
- jpttsn 9y ago
  As does wordless pointing and grunting. Natural language is broad: language can be natural and still bad. So I don't think natural language belongs on the "endgame" pedestal.
  - ethbro 9y ago
    Here's why I put it there. Devices which feature functional natural language input can ditch virtual or physical keyboards and all the associated hardware. Which is a fundamentally more cost-effective design. Which is going to enable "smart" devices at price points that traditional screen-and-input devices can't compete at. And if only certain companies have strong enough AI and NL processing to enable this? That's going to reshape the market. (Yes, I'm looking at Apple)
    - jpttsn 9y ago
      That's a case for voice input, rather than natural language per se.
      - ethbro 9y ago
        Point, it is. I suppose the difference between those two will depend on future progress. It'll be curious if some languages' grammars are more amenable to NL.
- miguelrochefort 9y ago
  Natural languages (such as English) aren't the endgame because: - They don't constrain what you can and can't say based on context. A graphical user interface can adapt to your context by disabling and hidings options that aren't relevant or allowed. - They are linear and difficult to navigate. It's easier to navigate back and forward in time through a graph or tree than through a textual conversation. - They are textual, and lack the richness and feedback of sliders, color pickers, maps, 3D models, etc.
  - ethbro 9y ago
    While I'd agree with your points, I'd agree with them for me (and presumably for you). I have a feeling we and HN in general are outliers in terms of "looking at a UI, mentally decomposing its elements, applying common UI knowledge to them, experimenting until we get the action we want, and doing all the above in our first 10s with a new UI." As support, I would offer that despite the comments you've outlined above, normal people typically feel more comfortable performing an unfamiliar task via speech than GUI.
imh 9y ago
Can you expand on what you consider the AI UI problem and why you don't think anyone's working on it? If it's what I think it is, people are trying, but it's just hard and in some weird way not as flashy as "company X revolutionizes the world by making a 0.05% improvement in speech recognition task."

ethbro 9y ago

Natural language is always the endgame. Because natural language either removes the physical input system or frees up screen real estate.

kobeya 9y ago
Natural language can be incredibly imprecise though. I expect the ideal UI for AI will meet somewhere in the middle with something that combines the best of programming languages with the specificity and ease of use of natural language -- think "strongly typed natural language". I'd rather my command type-error than the machine go off and do what I say, not what I meant.
- ethbro 9y ago
  Step 1 was recognizing speech to text accurately. Step 2 is AI that is globally context- and subtext-aware. I expect Step 3 to be AI that is personally situation-, context-, and subtext-aware (IoT & personal info storage-enabled). And I think that's drastically going to improve accuracy. I can see some standardization of input (in the same way that GUIs evolved), but I think it will probably take the form of grammar structure & error behavior more than strongly typing questions.
  - dkersten 9y ago
    That will solve the AI understanding you, but still won't make me more precise and clear in my instructions. Yes, the AI can hopefully infer the rest, but even then, I can currently type faster than I can speak. Will the AI be able to infer enough of what I want that I can say substantially less and have the AI still get it right? At that stage, why not go all the way and remove me from the equation completely?
    - ethbro 9y ago
      On the typing angle, I'd argue the HN vs mass market point. Typing is still a skill we aren't born with. As much as it's now assumed in 1st world core demographics, there are a huge number of people for whom it isn't faster. And if you've ever watched a grandparent try and use Windows, there are a lot of other metaphors they miss too. I'm not saying NL recognition is going to be easy, but I am saying that getting there will be far more valuable than maybe the HN crowd assumes from self-reflection.
      - dkersten 9y ago
        It depends on the use case. If its a typical Google Home-style "turn on the lights", "look up this thing", "do my accounts" then I agree (although the AI may need to do a lot of prompting since people (at least me) often don't know what we really want). For less casual uses, I'm not sure. I guess there's a spectrum.
        ethbro 9y ago
        That's a good point on the prompting, and something I don't believe a lot of current systems do? E.g. "Do X" "Would you like me to X1 or X2?" I'd guess the complexity usually isn't there as (afaik) current systems tend to be single action endpoint rather than knowledge parsing the request.
        dkersten 9y ago
        Yeah I think you're right that current systems don't have the complexity yet. It's either simple tasks (turn on the lights, play music), search (find X for me) or menus. As complexity of tasks increases, I think it'll become more important.
ragebol 9y ago
Yet, I don't like to use e.g. Google Now when I'm not alone. Dictating a personal text message to my phone in public is not what I want.
- vthallam 9y ago
  Yep. that's why they allow both chat and voice as input to the Google assistant. Voice is only one way of accessing consumer AI
- ethbro 9y ago
  Granted, but I'd expect subvocalization with predictive text or something else to infill where necessary. Stronger AI can substitute for a lot of "magic."
jpttsn 9y ago
As does wordless pointing and grunting. Natural language is broad: language can be natural and still bad. So I don't think natural language belongs on the "endgame" pedestal.
- ethbro 9y ago
  Here's why I put it there. Devices which feature functional natural language input can ditch virtual or physical keyboards and all the associated hardware. Which is a fundamentally more cost-effective design. Which is going to enable "smart" devices at price points that traditional screen-and-input devices can't compete at. And if only certain companies have strong enough AI and NL processing to enable this? That's going to reshape the market. (Yes, I'm looking at Apple)
  - jpttsn 9y ago
    That's a case for voice input, rather than natural language per se.
    - ethbro 9y ago
      Point, it is. I suppose the difference between those two will depend on future progress. It'll be curious if some languages' grammars are more amenable to NL.
miguelrochefort 9y ago
Natural languages (such as English) aren't the endgame because: - They don't constrain what you can and can't say based on context. A graphical user interface can adapt to your context by disabling and hidings options that aren't relevant or allowed. - They are linear and difficult to navigate. It's easier to navigate back and forward in time through a graph or tree than through a textual conversation. - They are textual, and lack the richness and feedback of sliders, color pickers, maps, 3D models, etc.
- ethbro 9y ago
  While I'd agree with your points, I'd agree with them for me (and presumably for you). I have a feeling we and HN in general are outliers in terms of "looking at a UI, mentally decomposing its elements, applying common UI knowledge to them, experimenting until we get the action we want, and doing all the above in our first 10s with a new UI." As support, I would offer that despite the comments you've outlined above, normal people typically feel more comfortable performing an unfamiliar task via speech than GUI.

imh 9y ago

Can you expand on what you consider the AI UI problem and why you don't think anyone's working on it? If it's what I think it is, people are trying, but it's just hard and in some weird way not as flashy as "company X revolutionizes the world by making a 0.05% improvement in speech recognition task."