4 ms·

I saw the development of a chatbot based on the IBM Watson stuff. It was just like a phone tree map, except the system tries to guess the option selected based

I saw the development of a chatbot based on the IBM Watson stuff. It was just like a phone tree map, except the system tries to guess the option selected based on the intents found in the speech/text. Of course it got no adoption, except when forced on users to drive metrics. It was enormously expensive, it would have been cheaper to have a human on the other end.

SeanLuke 5y ago
> It was enormously expensive, it would have been cheaper to have a human on the other end. This sounds exactly like what someone might have said about early IBM mainframes.
- arpinum 5y ago
  Like early mainframes, not every task is a good fit for emerging tech.
- pintxo 5y ago
  While the mainframe is arguably still a success (after all, they still run an amazing number of core services in our society: banks, air-traffic, governmental stuff, ... [1]), it's hard to find any evidence for a successful Watson project [2]. [1] https://www.bmc.com/blogs/state-of-mainframe/ https://www.bmc.com/blogs/state-of-mainframe/ [2] https://www.nytimes.com/2021/07/16/technology/what-happened-ibm-watson.html https://www.nytimes.com/2021/07/16/technology/what-happened-...
  - seszett 5y ago
    Well I'm still grieving Chef Watson. It probably doesn't count as successful since it didn't make money (I suppose this is why it was retired) but it was great at suggesting recipes from whatever ingredients I had, and ingredient pairings I wouldn't have thought about. I still haven't found anything that works like it.
- josefx 5y ago
  Wikipedia seems to consider the IBM 701 as mainframe and from the customers and special numeric format it had I assume it was used to replace entire floors of human calculators that did nothing but compute the same calculation over and over again. It was also rented for a monthly fee, so the cost comparison was probably straight forward. I wouldn't be surprised if an advanced chat bot with science fiction level AI would replace entire floors of Microsoft call centers at some point in the far future.
thrower123 5y ago
In almost every case, it's better to just use a regular dumb dialog tree than to try using any of the whizz-bang NLP stuff. I also have some horror stories about Watson that I'd love to tell if they weren't under disclosure agreements... All of the advanced features like this that are built into chatbot products are only there because they look fancy when you give the demo. From a customer perspective, you only ever want to get connected to the second-level customer service agent that might possibly be able to tell you to do something you haven't already tried. And from the business side, they want to get as many interactions out of the funnel so that they don't need to be connected, and thus they can run their customer service desks even leaner.
- kaibee 5y ago
  My favorite phone tree implementation is the ones where it takes twice as long to hear all of the options because it has to tell you the keyword(s) and the equivalent number to press (since I guess they don't have enough faith in it otherwise).
2sk21 5y ago
I was one of the original developers on this project! It was called the Watson Assistant and was a completely different code base from the original Jeopardy code base. The dialog tree was actually fairly sophisticated in allowing for a lot of flexibility. However, I always had a concern hat developers would find it too complex to author dialogs. Apart from the dialog tree, the other components were a text classifier and an entity detection system. It has been several years since I quit IBM so I have no idea how it has evolved since then.
- chrisco255 5y ago
  That makes me wonder how much of "bad chatbot" implementations are just poorly designed dialog trees? It seems like a domain-specific chatbot could be really good, if designed properly, but it's probably as much of an art as it is a science and most implementations probably don't go deep enough.
  - arpinum 5y ago
    The initial dialog tree and year of “improvements” were done by IBM themselves. The main issue was that it was a vanity project for a stakeholder, which they used to pivot into a new job / career as a “thought leader”

SeanLuke 5y ago

> It was enormously expensive, it would have been cheaper to have a human on the other end. This sounds exactly like what someone might have said about early IBM mainframes.

arpinum 5y ago
Like early mainframes, not every task is a good fit for emerging tech.
pintxo 5y ago
While the mainframe is arguably still a success (after all, they still run an amazing number of core services in our society: banks, air-traffic, governmental stuff, ... [1]), it's hard to find any evidence for a successful Watson project [2]. [1] https://www.bmc.com/blogs/state-of-mainframe/ https://www.bmc.com/blogs/state-of-mainframe/ [2] https://www.nytimes.com/2021/07/16/technology/what-happened-ibm-watson.html https://www.nytimes.com/2021/07/16/technology/what-happened-...
- seszett 5y ago
  Well I'm still grieving Chef Watson. It probably doesn't count as successful since it didn't make money (I suppose this is why it was retired) but it was great at suggesting recipes from whatever ingredients I had, and ingredient pairings I wouldn't have thought about. I still haven't found anything that works like it.
josefx 5y ago
Wikipedia seems to consider the IBM 701 as mainframe and from the customers and special numeric format it had I assume it was used to replace entire floors of human calculators that did nothing but compute the same calculation over and over again. It was also rented for a monthly fee, so the cost comparison was probably straight forward. I wouldn't be surprised if an advanced chat bot with science fiction level AI would replace entire floors of Microsoft call centers at some point in the far future.

thrower123 5y ago

In almost every case, it's better to just use a regular dumb dialog tree than to try using any of the whizz-bang NLP stuff. I also have some horror stories about Watson that I'd love to tell if they weren't under disclosure agreements... All of the advanced features like this that are built into chatbot products are only there because they look fancy when you give the demo. From a customer perspective, you only ever want to get connected to the second-level customer service agent that might possibly be able to tell you to do something you haven't already tried. And from the business side, they want to get as many interactions out of the funnel so that they don't need to be connected, and thus they can run their customer service desks even leaner.

kaibee 5y ago
My favorite phone tree implementation is the ones where it takes twice as long to hear all of the options because it has to tell you the keyword(s) and the equivalent number to press (since I guess they don't have enough faith in it otherwise).

2sk21 5y ago

I was one of the original developers on this project! It was called the Watson Assistant and was a completely different code base from the original Jeopardy code base. The dialog tree was actually fairly sophisticated in allowing for a lot of flexibility. However, I always had a concern hat developers would find it too complex to author dialogs. Apart from the dialog tree, the other components were a text classifier and an entity detection system. It has been several years since I quit IBM so I have no idea how it has evolved since then.

chrisco255 5y ago
That makes me wonder how much of "bad chatbot" implementations are just poorly designed dialog trees? It seems like a domain-specific chatbot could be really good, if designed properly, but it's probably as much of an art as it is a science and most implementations probably don't go deep enough.
- arpinum 5y ago
  The initial dialog tree and year of “improvements” were done by IBM themselves. The main issue was that it was a vanity project for a stakeholder, which they used to pivot into a new job / career as a “thought leader”