4 ms·

While the tech is constantly advancing, I have yet to use one in anger that fully solved my problem outside of giving me information that's already easy to find

While the tech is constantly advancing, I have yet to use one in anger that fully solved my problem outside of giving me information that's already easy to find. I think the only place it makes sense is in very very specific use-cases. Think about, say, a dental clinic. I would guess that if you put a human on chat support for a year in that setting, you'd see common themes emerging in customer problems. Maybe you develop a good library of canned responses and you find they work for 70 - 80% of queries, great. Then you could probably build a good chat bot to handle questions and handle cancelling/rescheduling appointments.

ChefboyOG 5y ago
I think this gets at what the actual value prop (not the marketed one) of chatbots really is: scale. There will not, in the foreseeable future, be a model/ensemble/pipeline capable of conversing at the level of a human representative. Thinking of them as "humans, but cheaper" is destined to fail. There are, however, use-cases where communicating with a huge group of individuals on a one-on-one level is useful. For example, years ago a company called Mainstay (then AdmitHub) built a chatbot for Georgia State designed to reduce "summer melt," a phenomenon in which intending freshmen drop off between graduating high school and enrolling in classes. For this use case, getting human-level conversation wasn't as critical as effective performance on tasks like question answering. Their bot, Pounce, was credited with a ~20% decrease in summer melt. They have a proper study here: https://mainstay.com/case-study/how-georgia-state-university-supports-every-student-with-personalized-text-messaging/ https://mainstay.com/case-study/how-georgia-state-university... This is an example, in my opinion, of the sweet-spot where intelligent, individualized, but not-near-human-level communication is needed, at a scale that goes beyond what you could reasonably do manually without an enormous dedicated staff. Now, I don't know how numerous those opportunities are, or if they come anywhere close to the level of hype and funding chatbot companies have received. I just think that in a vacuum, there is a set of problems where they have value.

ChefboyOG 5y ago

I think this gets at what the actual value prop (not the marketed one) of chatbots really is: scale. There will not, in the foreseeable future, be a model/ensemble/pipeline capable of conversing at the level of a human representative. Thinking of them as "humans, but cheaper" is destined to fail. There are, however, use-cases where communicating with a huge group of individuals on a one-on-one level is useful. For example, years ago a company called Mainstay (then AdmitHub) built a chatbot for Georgia State designed to reduce "summer melt," a phenomenon in which intending freshmen drop off between graduating high school and enrolling in classes. For this use case, getting human-level conversation wasn't as critical as effective performance on tasks like question answering. Their bot, Pounce, was credited with a ~20% decrease in summer melt. They have a proper study here: https://mainstay.com/case-study/how-georgia-state-university-supports-every-student-with-personalized-text-messaging/ https://mainstay.com/case-study/how-georgia-state-university... This is an example, in my opinion, of the sweet-spot where intelligent, individualized, but not-near-human-level communication is needed, at a scale that goes beyond what you could reasonably do manually without an enormous dedicated staff. Now, I don't know how numerous those opportunities are, or if they come anywhere close to the level of hype and funding chatbot companies have received. I just think that in a vacuum, there is a set of problems where they have value.