3 ms·
This is the key: > The system is only allowed to answer questions if it calculates that it is 97 percent or more confident in its answer. Knowing when you don
by ars 11y ago
This is the key:
> The system is only allowed to answer questions if it calculates that it is 97 percent or more confident in its answer.
Knowing when you don't know the answer is something that has been lacking from AI I've seen. Of course the problem is that not all AI has the option of deferring to a human.
Anyone know what percent of questions the AI answered, vs redirected?
- jobu 11y ago> Anyone know what percent of questions the AI answered, vs redirected? That's actually a very good question. The article doesn't give many details other than: "There are many questions Jill can’t handle." For comparison, here's are the highlights of a study from 2014 on the mobile assistants (Siri, Cortana, Google): http://searchengineland.com/google-now-beats-siri-cortana-direct-answers-study-205388 http://searchengineland.com/google-now-beats-siri-cortana-di... Google was able to offer "enhanced" answers (i.e. more than a web search) 58% of the time. Siri was at 29% and Cortana was at 20%. Siri improved some with iOS9, but not enough to beat google: http://www.recode.net/2015/9/20/11618696/how-intelligent-is-siri-we-put-her-through-a-few-tests http://www.recode.net/2015/9/20/11618696/how-intelligent-is-... From the actual study: ... we took 3086 different queries and compared them across all three platforms. These were not random queries. In fact, they were picked because we felt they were likely to trigger a knowledge panel. ( https://www.stonetemple.com/great-knowledge-box-showdown/ https://www.stonetemple.com/great-knowledge-box-showdown/ )
- ars 11y ago> That's actually a very good question. The article doesn't give many details other than: "There are many questions Jill can’t handle." I emailed and asked. If I get a reply I'll post it here.
- deckar01 11y agoIt would be nice if Jill Watson could respond to your email.
- tracker1 11y agoI really appreciate the enhanced answers.. "OK Google" has become my go to tool for a lot of things... "OK Google... what is 5 ounces in grams", or the inverse. It's almost invaluable, and could definitely make certain classes of applications (such as a calorie tracking app) far easier to use.
- Florin_Andrei 11y agoSo, I'm guessing right now Jill performs about as well as a human TA who is pretty new on the job. Nothing wrong with that, of course.
- deleted 11y ago[deleted]
- deleted 11y ago[deleted]
- lpage 11y ago> Knowing when you don't know the answer is something that has been lacking Definitely, and I'm not sure why that space isn't getting more attention. For RL there's KWIK (knows what it knows) http://www.research.rutgers.edu/~lihong/pub/Li08Knows.pdf http://www.research.rutgers.edu/~lihong/pub/Li08Knows.pdf [pdf]. A large class of problems (the robo TA being one of them) lend themselves very well to AI with error bounds, for lack of a better term.
- cubano 11y agoThat constraint alone makes the AI approx. 92.3% more wise then most humans.
- dghughes 11y agoAnytime I see anything AI related it seems context is the key otherwise answers are all over the place. It's the same for everyday life context seems to be the driver of humans.