4 ms·

This is why I'm very skeptical of this current craze for chat bots. It seems to me that they're really only useful if they can infer a wide range of intent from

This is why I'm very skeptical of this current craze for chat bots. It seems to me that they're really only useful if they can infer a wide range of intent from the user's input, and that seems like it's still very much an unsolved problem. Otherwise they're just glorified menu systems.

Animats 11y ago
This is easy to do badly and very hard to do well. It's not just about inferring intent. Most discourse systems have little or no model of the consequences of their actions. Systems that can act on the user's behalf need to know if they're potentially making a big mistake or a little one, as guidance of when to ask for clarification. This requires some degree of "common sense". AI systems still suck at that. Systems that actually do something need to be much better at this than ones that just answer questions. There needs to be some cost/benefit analysis of whether it's necessary to repeat back info and ask for confirmation. Without that, the system will either ask for confirmation too much, or screw up too much. The system needs to have a model of how clear the user is in their intent. Pizza ordering is relatively low-risk. Travel reservations are a higher-stakes item. Medical is a long way off. As I wrote last time, I recommend watching "The Devil Wears Prada", the scenes of Andy's first day as Miranda's personal assistant. That's the level of performance you want.
- cageface 11y ago
  Good point. I've been thinking so much about the difficulty of inferring intent I hadn't really considered the consequences of getting it wrong. The more responsible we allow these chat bots to be the greater the risks become. This is why I think that eventually people will bring this technology up to a useful level it's only going to be the kinds of companies that can make huge time and money investments and have access to huge datasets. Not startup territory really.
- Animats 11y ago
  For those of you who haven't seen "The Devil Wears Prada", here's Miranda giving orders to her assistant: I need 10 or 15 skirts from Calvin Klein. -What kind of skirts do you… -Please bore someone else with your questions. And make sure we have Pier 59 at 8:00 a.m. Tomorrow. Remind Jocelyn I need to see a few of those satchels that Marc is doing in the pony. And then tell Simone I'll take Jackie if Maggie isn't available. -Did Demarchelier confirm? -D-Did D-Demarchel… Demarchelier. Did he… Get him on the phone. Uh, o… okay. -And, Emily? -Yes? That's all.
natrius 11y ago
I expect glorified menu systems to be fantastic. I'd like to replace every phone call I make to a corporation with a glorified menu system.
- pjc50 11y ago
  Indeed. I'd like to see this the other way round: a nice web page representing a business's phone tree and scripts, some forms I can fill the answers into, and then I can have a smart text-to-speech robot wait in the queue and answer the questions on my behalf.
  - jsmeaton 11y ago
    Your idea is mostly right. Instead of a robot waiting in queue though you should receive the call with the information you already entered presented to the agent. Some companies do this.

Animats 11y ago

This is easy to do badly and very hard to do well. It's not just about inferring intent. Most discourse systems have little or no model of the consequences of their actions. Systems that can act on the user's behalf need to know if they're potentially making a big mistake or a little one, as guidance of when to ask for clarification. This requires some degree of "common sense". AI systems still suck at that. Systems that actually do something need to be much better at this than ones that just answer questions. There needs to be some cost/benefit analysis of whether it's necessary to repeat back info and ask for confirmation. Without that, the system will either ask for confirmation too much, or screw up too much. The system needs to have a model of how clear the user is in their intent. Pizza ordering is relatively low-risk. Travel reservations are a higher-stakes item. Medical is a long way off. As I wrote last time, I recommend watching "The Devil Wears Prada", the scenes of Andy's first day as Miranda's personal assistant. That's the level of performance you want.

cageface 11y ago
Good point. I've been thinking so much about the difficulty of inferring intent I hadn't really considered the consequences of getting it wrong. The more responsible we allow these chat bots to be the greater the risks become. This is why I think that eventually people will bring this technology up to a useful level it's only going to be the kinds of companies that can make huge time and money investments and have access to huge datasets. Not startup territory really.
Animats 11y ago
For those of you who haven't seen "The Devil Wears Prada", here's Miranda giving orders to her assistant: I need 10 or 15 skirts from Calvin Klein. -What kind of skirts do you… -Please bore someone else with your questions. And make sure we have Pier 59 at 8:00 a.m. Tomorrow. Remind Jocelyn I need to see a few of those satchels that Marc is doing in the pony. And then tell Simone I'll take Jackie if Maggie isn't available. -Did Demarchelier confirm? -D-Did D-Demarchel… Demarchelier. Did he… Get him on the phone. Uh, o… okay. -And, Emily? -Yes? That's all.

natrius 11y ago

I expect glorified menu systems to be fantastic. I'd like to replace every phone call I make to a corporation with a glorified menu system.

pjc50 11y ago
Indeed. I'd like to see this the other way round: a nice web page representing a business's phone tree and scripts, some forms I can fill the answers into, and then I can have a smart text-to-speech robot wait in the queue and answer the questions on my behalf.
- jsmeaton 11y ago
  Your idea is mostly right. Instead of a robot waiting in queue though you should receive the call with the information you already entered presented to the agent. Some companies do this.