21 ms·
A professor built a chatbot to be his teaching assistant
- alexcaps 10y agoThis sounded cool until, "Now Goel is forming a business to bring the chatbot to the wider world of education.” // are all educators just trying to create companies these days?
- 6stringmerc 10y agoIn my honest opinion, innovating in the Educational sphere and showing value to one or more groups - educators, students, administrative tasks - should really be rewarded fiscally. An institution won't really do that, except in the long-term by way of job stability and benefits (all subject to modern fiscal pressures). So, while it might seem the case, I think this avenue is perfectly reasonable. Now, educators who write texts in concert with publishers and then somehow coerse students into buying brand new copies every semester, that's what I'd call profiteering. Not so highly esteemed to me. But, as noted, the profession is a tradeoff a lot of times.
- spyspy 10y agoThe world of education could use some real technological disruption. Giving students iPads and smartboards doesn't cut it if they're just doing the same things on a new platform.
- hrkristian 10y agoEducation could use such an overhaul to be honest. Look at internet resources like Khan Academy and how they're proving the merits of mass-distributing a well-functioning formula for teaching. As far as this bot is concerned it seems to do one thing very well: Reducing the costs of offering a teaching method for those who require "human interaction" to learn optimally (spoonfeeding if you want).
- lpage 10y agoFor anyone interested, the class, which covers interactive intelligence and knowledge based AI, is freely available at https://www.udacity.com/course/knowledge-based-ai-cognitive-systems--ud409 https://www.udacity.com/course/knowledge-based-ai-cognitive-... . It's a great starting point if you're looking to automate the sort of human intelligence tasks that don't lend themselves well to the traditional searching/planning/proving/minimax/retrieval route that underpins most AI. I took it before they rolled out the chat bot TA, unfortunately.
- jonkiddy 10y agoI took that course a year ago and thoroughly enjoyed it. Writing an AI agent to take IQ tests was a rewarding experience. I'm sure I asked/answered more than a dozen questions during that semester within the Piazza platform and likely unknowingly contributed to Jill Watson's training.
- patrickbolle 10y agothanks for the link ! question though does anybody have any good resources for doing something similar to this in javascript? I know a small amount of java, no python, and want to brush up my js skills anyways.
- vinodkd 10y agoIIRC there was nothing specific to java or python, except for using opencv for the last exercise to visually solve ravens matrices. You could do the same with nodejs, I'd presume. It might be easier too, considering most of the code involved manipulating dynamic data structures.
- algorithmmonkey 10y agoAlso took it before the chat bot... :( If you are thinking about a masters in CS, you must check out the OMSCS program. The cost to value ratio is completely off the charts! "total program cost of about $6,600 over five terms"
- mdorazio 10y ago"Goel plans to use Jill again in a class this fall, but will likely change its name so students have the challenge of guessing which teaching assistant isn't human." This will be a fantastic Turing test, at least as far as basic question understanding and natural response formation.
- lordnacho 10y agoHmm, does it count to use things that are not so much to do with the course? For example, if a guy is able to answer questions at all times of day, really fast, they're either an AI or Jon Skeet. Also, real TAs have a life outside of the course. What will an AI say when you ask them whether they enjoyed last night's episode of Game of Thrones?
- tfm 10y agoThe software worked remarkably well in this instance because the system was well constrained, and the students weren't on the lookout for it. If they do start quizzing their new unseen TA (Dr Colin V Olutionalnet) on random trivia it could presumably deflect the query or have a few canards pre-programmed.
- Coding_Cat 10y agoBut likewise, if I were a TA for that course I'd probably try to act like a bot just to confuse the students.
- jandrese 10y ago"I am very busy, please stick to the course material." I wonder if the professor has it configured to learn from the student's questions? Will the bot turn into a promiscuous neo-nazi a week into the class?
- exodust 10y agoAlready posted in original form: https://news.ycombinator.com/item?id=11688061 https://news.ycombinator.com/item?id=11688061 I don't know why SMH would take a Washington Post story and change the title to something stupid, while keeping everything else the same, but that's what they've done. Wait.. there's one more difference. A pathetic picture of "an AI". "an AI"? Seriously SMH, just use the original title. Your editorial blundering is embarrassing, and could easily be replaced with "an AI" tasked with re-wording headlines.
- oarsinsync 10y agoThanks for that, the original article was a much better read.
- exodust 10y agothe original has the sample interaction with students, so its definitely worth switching, otherwise its the same copy apart from butchered title. SMH is a bottom of barrel source. Changing a story title to something more click-baity, sums up the entire Fairfax network and Australian news media in general.
- dang 10y agoOk, we changed the URL to that from http://www.smh.com.au/technology/innovation/professor-reveals-to-students-that-his-assistant-was-an-ai-all-along-20160513-gou6us.html http://www.smh.com.au/technology/innovation/professor-reveal....
- wyldfire 10y agoThat's stunning. I studied AI in undergrad and I just didn't give it enough of my focus at the time. Little did I know that in my lifetime I'd see real world evidence of computers passing the Turing Test with flying colors. Of course, Turing was right. What's outside of the set of computable sets? Not much! I recently read an article headline that talked about creeps using AI to stalk porn stars. It's interesting to think that as computation power continues to grow, there's a huge potential for evil uses of AI. All of the things we see and hear and consider "public" have little or no nefarious uses to other humans. But what about a computer that has human-like capabilities and inexhaustible computation resources? The sci-fi scenarios of conscious AI making decisions to save humanity from itself are not what I fear. The evil humans sitting at the helm of powerful AI is what I fear. A lot of society's norms are predicated on humans having private thoughts and telling small and large lies as appropriate. What happens when AI-augmented humans know when you're lying? The future is so exciting and so terrifying.
- eternalban 10y ago> What's outside of the set of computable sets? Human intution can jump. Mechanical deduction must expend energy. Let me give you an example: Dmitri Mendeleev goes to bed, has a dream, and then delivers the periodic table. Can your machine do that?
- nairboon 10y agoWhere is the jump? There was a lot of computation preceding his dream, how is that different from deduction?
- lordnacho 10y agoThe periodic table is the elements ordered such that common features occur together. You'd think some sort of clustering algorithm would detect this. A line of elements that almost never react with anything. A bunch of things that gain one electron, some that lose one. And so on.
- eternalban 10y ago
- lordnacho 10y agoMakes sense as a use case. TAs must hear the same questions over and over again from different students, both practical things like "what homework is there" and reference stuff like "What does the line above the X mean?" (No pun intended!) I wonder how good it is when people don't know what they're asking. This used to happen all the time when I was in tutorials. "I don't get why there's two transistors in a chain". "What's the significance of the process being adiabatic?" If you can crack that, education will be changed forever, massively.
- Bartweiss 10y agoThis is definitely an impressive result, but I'm pretty sure it was made easier by the opposite pattern. I've had a couple of TAs who I strongly suspected of being chatbots, even when I was talking to them in person. I'm still not 100% convinced they were human.
- SilasX 10y agoYep, and it's not like you want to make such an accusation when you're in that position. In fact, that's one way to pass the Turing test: any judge that suggests a subject is bot, experiences a permanent loss of social status regardless of correctness!
- tfm 10y agoMore human than human is our motto ... Next step will be to replace the prof, and then have the students replaced by surprisingly attentive interrogative expert systems, and the cycle will be complete.
- itgoon 10y agoReminds me of a scene in the movie "Real Genius". The class was empty of people, was just tape recorders listening to another tape recorder set up by the professor.
- HillaryBriss 10y agoAbsolutely. And, once that's set up, our banking sector and government should profit from hefty educational loans made to all those student bots.
- qb45 10y agoYeah, there exist humans who fail the Turing test. Another thing is foreign students and teachers who, even if otherwise competent, quickly get one used to language comparable with machine output. I think I may have been guilty of that at one time :)
- Kinnard 10y ago
- ikeboy 10y ago>Goel and his teaching assistants receive more than 10,000 questions a semester from students on the course's online forum. Is this normal?
- BorgHunter 10y agoI think that figure likely includes all contributions, not just questions. I took this same class in Summer 2015, and there are 1036 posts[0] and 11074 contributions[1] on the online forum for that class. The number of students is similar (370 in my class, 302 last semester). [0] posts are questions and notes [1] contributions are posts, responses, edits, followups, and comments to followups (i.e., everything)
- burkaman 10y agoNot 10,000 questions, 10,000 messages: http://www.news.gatech.edu/2016/05/09/artificial-intelligence-course-creates-ai-teaching-assistant http://www.news.gatech.edu/2016/05/09/artificial-intelligenc... That article says ~300 students, 30 posts each seems pretty normal.
- frozenport 10y agoNo! I've done a 300 person electronics class and we've gotten about 300 questions with a handful of people who ask the bulk of our questions.
- ikeboy 10y agoSo the original article: 1. Didn't mention it was an online course 2. Didn't mention there were 300 students 3. Claimed it was questions instead of posts
- burkaman 10y agoI don't think 1 and 2 are very important for the story, and the questions/posts thing seems like an honest mistake. No harm done. The meaning is basically the same, since the bot can presumably reply to any post, not just a top level question.
- ars 10y agoThis is the key: > The system is only allowed to answer questions if it calculates that it is 97 percent or more confident in its answer. Knowing when you don't know the answer is something that has been lacking from AI I've seen. Of course the problem is that not all AI has the option of deferring to a human. Anyone know what percent of questions the AI answered, vs redirected?
- jobu 10y ago> Anyone know what percent of questions the AI answered, vs redirected? That's actually a very good question. The article doesn't give many details other than: "There are many questions Jill can’t handle." For comparison, here's are the highlights of a study from 2014 on the mobile assistants (Siri, Cortana, Google): http://searchengineland.com/google-now-beats-siri-cortana-direct-answers-study-205388 http://searchengineland.com/google-now-beats-siri-cortana-di... Google was able to offer "enhanced" answers (i.e. more than a web search) 58% of the time. Siri was at 29% and Cortana was at 20%. Siri improved some with iOS9, but not enough to beat google: http://www.recode.net/2015/9/20/11618696/how-intelligent-is-siri-we-put-her-through-a-few-tests http://www.recode.net/2015/9/20/11618696/how-intelligent-is-... From the actual study: ... we took 3086 different queries and compared them across all three platforms. These were not random queries. In fact, they were picked because we felt they were likely to trigger a knowledge panel. ( https://www.stonetemple.com/great-knowledge-box-showdown/ https://www.stonetemple.com/great-knowledge-box-showdown/ )
- eternalban 10y ago"Education is such a huge priority for the entire human race." Half a century into this current iteration of the game called 'Life as a Human on planet Earth' I have come to the conclusion that our primary focus must be Education and Mental Health. Take care of these and we may yet see our full positive potential.
- collyw 10y agoRather than inventing sophisticated chat bots, couldn't he have just had a well organised web page? If the question were where is assignment two / when is assignment due. The examples given seem like a "high tech" (possibly unreliable) solution to a low tech problem.
- dimino 10y agoI could reasonably ride a horse to work every day, it's only 2 miles.
- adekok 10y agoAnd where would you get said horse? Where would you keep it? What would you feed it? Who would take care of it? Students should be capable of reading text. If they can't, they belong in kindergarten, not university.
- betenoire 10y agoI see it as a "high tech" interface to the same solution.
- lucb1e 10y agoExcept you have to publicly post your question before receiving search results. A search function is a lot faster and you can do a few attempts.
- srtjstjsj 10y agoActual URL path: /this-professor-stunned-his-students-when-he-revealed-the\ -secret-identity-of-his-teaching-assistant/ HN can do better than clickbait sites like the WaPo.
- TillE 10y agoI've never read much of the Washington Post, so I can't say whether there's actually been a decline since Bezos bought them. But there's been a remarkable quantity of terrible reporting from them lately, so at least we can say it's not very good under his ownership.
- cdnsteve 10y agoAnyone have technical details about this story? What kind of bot used, languages, algorithms, etc?
- orcasauce 10y agoIBM Watson Here's a better article that was linked a few days ago: http://www.wsj.com/articles/if-your-teacher-sounds-like-a-robot-you-might-be-on-to-something-1462546621 http://www.wsj.com/articles/if-your-teacher-sounds-like-a-ro...
- pesenti 10y agoIt's using this: http://www.ibm.com/smarterplanet/us/en/ibmwatson/engagement_advisor.html http://www.ibm.com/smarterplanet/us/en/ibmwatson/engagement_... but if you are smart you can reproduce a lot of the functionality using both http://www.ibm.com/smarterplanet/us/en/ibmwatson/developercloud/nl-classifier.html http://www.ibm.com/smarterplanet/us/en/ibmwatson/developercl... and http://www.ibm.com/smarterplanet/us/en/ibmwatson/developercloud/dialog.html http://www.ibm.com/smarterplanet/us/en/ibmwatson/developercl.... We are also coming up this quarter with a fully integrated self-service API that will make doing all this much easier (especially the dialog part). Stay posted.
- lucb1e 10y agoI can't help feeling a big FAQ page with a search function would be a lot more useful. Especially if the entries contain aliases for common search terms, like "length" on the entry of "word count", it should cover everything. Now this is a forum where others can chip in as well, but the FAQ page could prevent you from having to post in the first place. And if you don't have to post publicly to get an answer from the chat bot (because it's a FAQ page and not a bot) you can do 20 searches before deciding it's not in there.
- BrandonSmithJ 10y agoSpeaking from experience from being in this class, the unfortunate truth is many people don't bother to read the documents for assignments (thoroughly, at any rate). This one wasn't as bad as another one I took, but many people will ask questions which are easily answered by looking at the materials they're given. That's essentially the area in which this AI excels.
- sandaru1 10y agoAnyone know whether they used watson APIs to create this bot or used the original watson codebase (with IBM collaboration)?
- pesenti 10y agoIt's using this: http://www.ibm.com/smarterplanet/us/en/ibmwatson/engagement_advisor.html http://www.ibm.com/smarterplanet/us/en/ibmwatson/engagement_... but if you are smart you can reproduce a lot of the functionality using both http://www.ibm.com/smarterplanet/us/en/ibmwatson/developercloud/nl-classifier.html http://www.ibm.com/smarterplanet/us/en/ibmwatson/developercl... and http://www.ibm.com/smarterplanet/us/en/ibmwatson/developercloud/dialog.html http://www.ibm.com/smarterplanet/us/en/ibmwatson/developercl.... We are also coming up this quarter with a fully integrated self-service API that will make doing all this much easier (especially the dialog part). Stay posted.
- thaw13579 10y agoI wonder how many of the responses were "check the syllabus"
- forrestbrazeal 10y agoI was a student in this class, and had at least one question on the forum answered (correctly) by Jill. I can see this technology being hugely useful for teachers who conduct large lecture classes in any subject on a regular basis. That said, I'd be even more excited if Jill had the ability to synthesize new answers to questions through some type of case-based reasoning. This would require Jill receiving feedback on "her" answers, which might mean the students would have to know "who" "she" was in advance. (Sorry, got lost in the quotes.) Right now, Jill is essentially an automated FAQ-retrieval bot.
- ilyaeck 10y agoDo you know what kind of classification method it is using at the moment?
- forrestbrazeal 10y agoNot exactly. I know it is built on an academic flavor of Watson, though.
- deleted 10y ago[deleted]
- SilasX 10y agoIf you had some reddit/HN-style forum where the questions get replies and votes, that would be valuable information for what counts as a "good answer".
- jerryhuang100 10y agoin a way i'm not sure why there is still no parents or students complaining they're kind of being 'defrauded' for a chatbot while paying high tuition (cue discover card tv ad.) one thing for sure is that there are a lot of undergrad or grad students around looking for a part-time job and teaching experience. i know this is a great achievement in AI with huge potential in open course, and great for grant application, but on the very other end is someone paying with an expectation of a human interaction & care. for years i've seen a lot of complaints about foreign human TAs.
- pesenti 10y agoFor those wondering what technology is being used. It is using this Watson product: http://www.ibm.com/smarterplanet/us/en/ibmwatson/engagement_advisor.html http://www.ibm.com/smarterplanet/us/en/ibmwatson/engagement_.... We are in the process of revamping it and making it a self-service API that will be released this quarter in http://ibm.com/watsondevelopercloud http://ibm.com/watsondevelopercloud. Some of the core functionality is already available in http://www.ibm.com/smarterplanet/us/en/ibmwatson/developercloud/nl-classifier.html http://www.ibm.com/smarterplanet/us/en/ibmwatson/developercl... and http://www.ibm.com/smarterplanet/us/en/ibmwatson/developercloud/dialog.html http://www.ibm.com/smarterplanet/us/en/ibmwatson/developercl... in case you don't want to wait.
- xufi 10y agoPretty amazing. Considering that I've only really taken 1 online class so far excluding tutorials that I've done. I wonder if one day how this would be used for real responses for a real professor someday
- deepGem 10y agoI wonder what the 'layers of decision making software' on top of Watson mean. Are these rule engines ?
- zeeZ 10y agoThe site comes with an obnoxious "enter your email" overlay that won't go away and messes up scrolling when you block it. They do accept @washpost.com addresses though, so...
- mathheaven 10y agoThe next step is to build a chatbot to replace the professor at teaching. If someone is not going to enjoy the fruits why are you going to supply the knowledge base for a chatbot that is going to replace you?